Optical tracking system, and method for calculating posture and location of marker part in optical tracking system

ABSTRACT

An optical tracking system includes a marker part, an image forming part, and a processing part. The marker part includes a pattern having particular information and a first lens. First and second image forming parts include second and third lenses and first and second image forming units. The processing part determines the posture of the marker part from a first coordinate conversion formula between a coordinate on the pattern surface of a pattern and first pixel coordinate on a first image of the pattern, and a second coordinate conversion formula between a coordinate on the pattern surface of the pattern and second pixel coordinate on a second image of the pattern, the second coordinate conversion formula including the rotation conversion between the first pixel coordinate and the second pixel coordinate and tracks the marker part by using the posture of the marker part.

TECHNICAL FIELD

The present invention relates to an optical tracking system and a methodfor calculating the posture and location of a marker part of the opticaltracking system. More particularly, the present invention relates to anoptical tracking system and a method for calculating the posture andlocation of a marker part of the optical tracking system by usingpattern information.

BACKGROUND ART

In general, an optical tracking system is used to track the position ofa predetermined object. For example, the optical tracking system may beutilized to track a target in real time in equipment, such as a surgicalrobot.

The optical tracking system generally includes a plurality of markersattached to a target and image forming units for forming images by usinglight emitted by the markers, and mathematically calculates informationacquired from the image forming units to thereby obtain positioninformation or the like.

However, the conventional optical tracking system includes a pluralityof markers, which increases the size of the equipment, and may be thusinappropriate in the case of tracking that requires fine precision.

Therefore, an optical tracking system, which can track the markersaccurately and easily while simplifying the markers, is required.

SUMMARY

Therefore, an aspect of the present invention is to provide an opticaltracking system that can track markers accurately and easily whilesimplifying the markers.

Another aspect of the present invention is to provide a method ofcalculating the posture and location of a marker part of an opticaltracking system that can be applied to the optical tracking systemabove.

An optical tracking system, according to an exemplary embodiment of thepresent invention, may include: a marker part that is configured toinclude a pattern that has particular information and a first lens thatis spaced apart from the pattern and has a first focal length; a firstimage forming part that is configured to include a second lens that hasa second focal length and a first image forming unit that is spacedapart from the second lens and on which a first image of the pattern isformed by the first lens and the second lens; a second image formingpart that is configured to include a third lens that has a third focallength and a second image forming unit that is spaced apart from thethird lens and on which a second image of the pattern is formed by thefirst lens and the third lens; and a processing part that is configuredto determine a posture of the marker part from a first coordinateconversion formula between a coordinate on a pattern surface of thepattern and a first pixel coordinate on the first image of the patternand from a second coordinate conversion formula between the coordinateon the pattern surface of the pattern and a second pixel coordinate onthe second image of the pattern, the second coordinate conversionformula including a rotation conversion between the first pixelcoordinate and the second pixel coordinate, and is configured to trackthe marker part.

In an embodiment, the processing part may acquire: the first conversionmatrix that converts a first coordinate corresponding to a coordinate onthe pattern surface of the pattern to a second coordinate correspondingto a three-dimensional coordinate for the first lens of the marker part;the second conversion matrix that converts a third coordinatecorresponding to a three-dimensional coordinate of the second coordinatefor the second lens to a fourth coordinate corresponding to the firstpixel coordinate on the first image of the pattern of the first imageforming part; the third conversion matrix that is the same as the firstconversion matrix and converts a fifth coordinate corresponding to acoordinate on the pattern surface of the pattern to a sixth coordinatecorresponding to a three-dimensional coordinate for the first lens ofthe marker part; and the fourth conversion matrix that converts aseventh coordinate corresponding to a three-dimensional coordinate ofthe sixth coordinate for the third lens to an eighth coordinatecorresponding to the second pixel coordinate on the second image of thepattern of the second image forming part, wherein the first coordinateconversion formula is defined to convert the first coordinate to thefourth coordinate while including the first conversion matrix and thesecond conversion matrix, and the second coordinate conversion formulais defined to convert the fifth coordinate to the eighth coordinatewhile including the third conversion matrix and the fourth conversionmatrix, and wherein the processing part acquires, from the firstcoordinate conversion formula and the second coordinate conversionformula, a first posture definition matrix that defines the posture ofthe marker part with respect to the first image forming part.

For example, the first coordinate conversion formula may be defined bythe following equation,

${s\begin{bmatrix}{lu}^{\prime} \\{lv}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A_{l} \rbrack\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{lu} \\{lv} \\1\end{bmatrix}}$

wherein {(lu,lv) denotes the first coordinate, (lu′,lv′) denotes thefourth coordinate, [C] denotes the first conversion matrix, [A_(l)]denotes the second conversion matrix, [R_(L)] denotes the first posturedefinition matrix, and s denotes a proportional constant}, and

the second coordinate conversion formula may be defined by the followingequation,

${s\begin{bmatrix}{ru}^{\prime} \\{rv}^{\prime} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{R} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{LR} \rbrack}\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}}}$

wherein {(ru,rv) denotes the fifth coordinate, (ru′,rv′) denotes theeighth coordinate, [C] denotes the third conversion matrix, [A_(r).]denotes the fourth conversion matrix, [R_(R)] denotes the second posturedefinition matrix for defining the posture of the marker part withrespect to the second image forming part, [R_(LR)] denotes a thirdposture definition matrix for defining a posture of the first imageforming part with respect to the second image forming part, and sdenotes a proportional constant}.

For example, the first conversion matrix and the third conversion matrixmay be defined by the following equation,

$\lbrack C\rbrack = \begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}$

wherein {(u_(c),v_(c)) denotes the coordinate of a center of thepattern, and f_(b) denotes the first focal length}, and

the second conversion matrix and the fourth conversion matrix may bedefined by the following equation,

$\lbrack A\rbrack = \begin{bmatrix}{- \frac{f_{c}}{pw}} & 0 & u_{c}^{\prime} \\0 & {- \frac{f_{c}}{ph}} & v_{c}^{\prime} \\0 & 0 & 1\end{bmatrix}$

wherein {(u′_(c),v′_(c)) denotes the pixel coordinate on the image ofthe pattern corresponding to the center of the pattern, f_(c) denotesthe second focal length in the case of the second conversion matrix anddenotes the third focal length in the case of the fourth conversionmatrix, pw denotes a width of a pixel of the first image of the patternin the case of the second conversion matrix and denotes a width of apixel of the second image of the pattern in the case of the fourthconversion matrix, and ph denotes a height of a pixel of the first imageof the pattern in the case of the second conversion matrix and denotes aheight of a pixel of the second image of the pattern in the case of thefourth conversion matrix}.

In an embodiment, the processing part may acquire the first conversionmatrix and the third conversion matrix by acquiring calibration valuesof u_(c), u_(c), and f_(b) from three or more photographed images, andmay acquire the second conversion matrix and the fourth conversionmatrix by acquiring calibration values of f_(c), pw, and ph by using theacquired data.

In an embodiment, the processing part may acquire a plurality of piecesof data on the first coordinate and the fourth coordinate and aplurality of pieces of data on the fifth coordinate and the eighthcoordinate, and may acquire the first posture definition matrix by thefollowing equation to which the plurality of pieces of the acquired dataare applied,

$\lbrack R_{L} \rbrack = {{{\begin{bmatrix}r_{11} & r_{12} & r_{13} \\r_{21} & r_{22} & r_{23} \\r_{31} & r_{32} & r_{33}\end{bmatrix}\begin{bmatrix}{LW}_{1} \\{RW}_{1} \\\vdots \\{LW}_{n} \\{RW}_{n}\end{bmatrix}}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}$ ${LW}_{i} = \begin{bmatrix}{\frac{f_{c}}{pw}{lu}_{i}} & {\frac{f_{c}}{pw}{lv}_{i}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )u_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )v_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}{lu}_{i}} & {\frac{f_{c}}{ph}{lv}_{i}} & {\frac{f_{c}}{ph}f_{b}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )u_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )v_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )f_{b}}\end{bmatrix}$ ${RW}_{i} = \begin{bmatrix}{{A\; 1r_{11}^{\prime}} + {B\; 1r_{31}^{\prime}}} & {{C\; 1r_{11}^{\prime}} + {D\; 1r_{31}^{\prime}}} & {{E\; 1r_{11}^{\prime}} + {F\; 1r_{31}^{\prime}}} \\{{A\; 1r_{12}^{\prime}} + {B\; 1r_{32}^{\prime}}} & {{C\; 1r_{12}^{\prime}} + {D\; 1r_{302}^{\prime}}} & {{E\; 1r_{12}^{\prime}} + {F\; 1r_{32}^{\prime}}} \\{{A\; 1r_{13}^{\prime}} + {B\; 1r_{33}^{\prime}}} & {{C\; 1r_{13}^{\prime}} + {D\; 1r_{33}^{\prime}}} & {{E\; 1r_{13}^{\prime}} + {F\; 1r_{33}^{\prime}}} \\{{A\; 2r_{11}^{\prime}} + {B\; 2r_{31}^{\prime}}} & {{C\; 2r_{11}^{\prime}} + {D\; 2r_{31}^{\prime}}} & {{E\; 2r_{11}^{\prime}} + {F\; 2r_{31}^{\prime}}} \\{{A\; 2r_{12}^{\prime}} + {B\; 2r_{32}^{\prime}}} & {{C\; 2r_{12}^{\prime}} + {D\; 2r_{302}^{\prime}}} & {{E\; 2r_{12}^{\prime}} + {F\; 2r_{32}^{\prime}}} \\{{A\; 2r_{13}^{\prime}} + {B\; 2r_{33}^{\prime}}} & {{C\; 2r_{13}^{\prime}} + {D\; 2r_{33}^{\prime}}} & {{E\; 2r_{13}^{\prime}} + {F\; 2r_{33}^{\prime}}}\end{bmatrix}$${{A\; 1} = {{- \frac{f_{c}}{pw}}{ru}_{i}}},{{B\; 1} = {{ru}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{C\; 1} = {{- \frac{f_{c}}{pw}}{rv}_{i}}},{{D\; 1} = {{rv}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{E\; 1} = {{- \frac{f_{c}}{pw}}f_{b}}},{{F\; 1} = {f_{b}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}}$${{A\; 2} = {{- \frac{f_{c}}{ph}}{ru}_{i}}},{{B\; 2} = {{ru}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{C\; 2} = {{- \frac{f_{c}}{ph}}{rv}_{i}}},{{D\; 2} = {{rv}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{E\; 2} = {{- \frac{f_{c}}{ph}}f_{b}}},{{F\; 2} = {{{f_{b}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}\lbrack R_{LR} \rbrack} = \begin{bmatrix}r_{11}^{\prime} & r_{12}^{\prime} & r_{13}^{\prime} \\r_{21}^{\prime} & r_{22}^{\prime} & r_{23}^{\prime} \\r_{31}^{\prime} & r_{32}^{\prime} & r_{33}^{\prime}\end{bmatrix}}}$

wherein {(lu₁,lv₁), . . . , (lu_(n),lv_(n)) denote data on the firstcoordinates, (lu′₁,lv′₁), . . . , (lu′_(n),lv′_(n)) denote data on thefourth coordinates, (lu′_(c),lv′_(c)) denotes the pixel coordinates onthe first image of the pattern corresponding to the center of thepattern, (ru₁rv₁), . . . , (ru_(n),rv_(n)) denote data on the fifthcoordinate, (ru′₁,rv′₁), . . . , (ru′_(n),rv′_(n)) denote data on theeighth coordinate, and (ru′_(c),rv′_(c)) denotes the pixel coordinate onthe second image of the pattern corresponding to the center of thepattern}.

In an embodiment, the processing part may determine the location of themarker part from the third coordinate conversion formula for the secondcoordinate and the fourth coordinate and from the fourth coordinateconversion formula for the sixth coordinate and the eighth coordinate,and may track the marker part by using the determined location of themarker part.

For example, the third coordinate conversion formula may be defined bythe following equation,

${s\begin{bmatrix}u_{1}^{\prime} \\v_{1}^{\prime} \\1\end{bmatrix}} = {{A_{L}\lbrack {I❘0} \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}$

wherein {(u′₁,v′₁) denotes the fourth coordinate, (X,Y,Z) denotes thesecond coordinate, [A_(L)] denotes the second conversion matrix, [I]denotes an identity matrix in a 3×3 form, [0] denotes a zero-matrix in a3×1 form, and s denotes a proportional constant}, and

the fourth coordinate conversion formula may be defined by the followingequation,

${s\begin{bmatrix}u_{2}^{\prime} \\v_{2}^{\prime} \\1\end{bmatrix}} = {{A_{R}\lbrack {R_{LR}❘T} \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}$

wherein {(u′₂,v′₂) denotes the eighth coordinate, (X,Y,Z) denotes thesixth coordinate, [A_(R)] denotes the fourth conversion matrix, [R_(LR)]denotes the third posture definition matrix in a 3×3 form, [T] denotes alocation conversion matrix in a 3×1 form, and s denotes a proportionalconstant}.

In an embodiment, the processing part may: acquire a first centralcoordinate and a second central coordinate, each of which is the centerof a field of view of the pattern that is photographed in the first andsecond image forming units, respectively; calibrate the locationconversion matrix between the first image forming part and the secondimage forming part by using the acquired central coordinates; andacquire the location of the marker part by using the calibrated locationconversion matrix.

In an embodiment, the processing part may acquire a scale factor bymeasuring the marker part at two or more locations, and may calibratethe location conversion matrix between the first image forming part andthe second image forming part by using the acquired scale factortogether with the acquired central coordinates.

According to an exemplary embodiment of the present invention, a methodfor calculating a posture and a location of a marker part of an opticaltracking system is provided, which includes a marker part that includesa pattern that has particular information and a first lens that isspaced apart from the pattern and has a first focal length, a firstimage forming part that includes a second lens that has a second focallength and a first image forming unit that is spaced apart from thesecond lens and on which a first image of the pattern is formed by thefirst lens and the second lens, and a second image forming part thatincludes a third lens that has a third focal length and a second imageforming unit that is spaced apart from the third lens and on which asecond image of the pattern is formed by the first lens and the thirdlens, and which calculates the posture of the marker part for trackingthe marker part. The method for calculating the posture and the locationof the marker part of the optical tracking system may include: acquiringa first conversion matrix that converts a first coordinate correspondingto a coordinate on a pattern surface of the pattern to a secondcoordinate corresponding to a three-dimensional coordinate for the firstlens of the marker part, the second conversion matrix that converts athird coordinate corresponding to a three-dimensional coordinates of thesecond coordinate for the second lens to a fourth coordinatecorresponding to a first pixel coordinate on the first image of thepattern of the first image forming part, the third conversion matrixthat is the same as the first conversion matrix and converts a fifthcoordinate corresponding to a coordinate on the pattern surface of thepattern to a sixth coordinate corresponding to a three-dimensionalcoordinate for the first lens of the marker part, and the fourthconversion matrix that converts a seventh coordinate corresponding to athree-dimensional coordinate of the sixth coordinate for the third lensto an eighth coordinate corresponding to a second pixel coordinate onthe second image of the pattern of the second image forming part; andacquiring a posture definition matrix for defining the posture of themarker part from a first coordinate conversion formula that converts thefirst coordinate to the fourth coordinate while including the firstconversion matrix and the second conversion matrix and from a secondcoordinate conversion formula that converts the fifth coordinate to theeighth coordinate while including the third conversion matrix and thefourth conversion matrix.

In an embodiment, the method for calculating the posture and thelocation of the marker part of the optical tracking system may furtherinclude: acquiring a first central coordinate and a second centralcoordinate, each of which is the center of a field of view of thepattern that is photographed in the first and second image formingunits, respectively; calibrating the location conversion matrix betweenthe first image forming part and the second image forming part by usingthe acquired central coordinates; and acquiring the location of themarker part by using the calibrated location conversion matrix.

In an embodiment, the method for calculating the posture and location ofa marker part of an optical tracking system may further includeacquiring a scale factor by measuring the marker part at two or morelocations before calibrating the location conversion matrix, wherein thelocation conversion matrix between the first image forming part and thesecond image forming part may be calibrated by using the acquired scalefactor together with the acquired data in the calibrating of thelocation conversion matrix.

According to the present invention, in the optical tracking system fortracking a marker part, the marker part can be miniaturized whileincluding a pattern of particular information to enable the tracking,and the posture and the location of the marker part can be determinedmore accurately by modeling the optical system of the marker part andthe image forming part with the coordinate conversion formula and byconfiguring the optical system in stereo. Therefore, it is possible toaccurately track the marker part by a simpler and easier method.

In addition, since the location of the marker part can be determinedmore accurately by modeling the marker part in a stereo structure, whichhas been miniaturized while including the pattern of particularinformation, it is possible to accurately track the marker part by asimpler and easier method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual diagram illustrating an optical tracking system,according to an embodiment of the present invention.

FIG. 2 is a flowchart schematically showing a problem-solving processthat is necessary for the processing part of the optical tracking systemof FIG. 1 in determining the posture of the marker part.

FIG. 3 is a flowchart illustrating a process of system modeling in theproblem-solving process of FIG. 2.

FIG. 4 is a conceptual diagram for explaining the process of systemmodeling in FIG. 3.

FIG. 5 is a flowchart illustrating a process of calibrating a secondconversion matrix in the problem-solving process of FIG. 2.

FIG. 6 is a flowchart illustrating a process of calibrating a firstconversion matrix in the problem-solving process of FIG. 2.

FIG. 7 is a flowchart illustrating an example of a process of acquiringa posture definition matrix in problem-solving process of FIG. 2.

FIG. 8 is a flowchart illustrating another example of a process ofacquiring a posture definition matrix in problem-solving process of FIG.2.

FIG. 9 is a flowchart illustrating a method of calculating the postureof a marker part of an optical tracking system, according to anembodiment of the present invention.

FIG. 10 is a conceptual diagram illustrating an optical tracking system,according to another embodiment of the present invention.

FIG. 11 is a flowchart schematically showing a problem-solving processthat is necessary for a processing part of the optical tracking systemof FIG. 10 to determine the posture of the marker part.

FIG. 12 is a flowchart illustrating a process of calibrating conversionmatrixes in the problem-solving process of FIG. 11.

FIG. 13 is a flowchart illustrating an example of a process of acquiringa posture definition matrix in the problem-solving process of FIG. 11.

FIG. 14 is a flowchart illustrating a method of calculating the postureof a marker part of an optical tracking system, according to anotherembodiment of the present invention.

FIG. 15 is a flowchart schematically showing a problem-solving processthat is necessary for a processing part of the optical tracking systemof FIG. 10 to determine the location of the marker part.

FIG. 16 is a conceptual diagram for explaining the process of systemmodeling in the problem-solving process of FIG. 15.

FIG. 17 is a flowchart illustrating a process of calibrating thelocation conversion matrix in the problem-solving process of FIG. 15.

FIG. 18 is a flowchart illustrating an example of a process of acquiringthe location of a marker part in the problem-solving process of FIG. 15.

FIG. 19 is a flowchart illustrating another example of a process ofacquiring the location of a marker part in the problem-solving processof FIG. 15.

FIG. 20 is a flowchart illustrating a method of calculating the locationof a marker part of an optical tracking system, according to anembodiment of the present invention.

DETAILED DESCRIPTION

Although the present invention may be variously modified and may have avariety of forms, particular embodiments will be shown in the drawingsand will be described in the specification. However, this is notintended to limit the present invention to particular disclosed forms,and it should be understood that the particular embodiments mayencompass all modifications, equivalents, and substitutes that areincluded in the spirit and scope of the present invention.

Although the terms “first” or “second” may be used to describe variouselements, the elements are not limited to the terms. The terms abovewill be used only to distinguish one element from other elements. Forexample, the first element may be named as the second element withoutdeparting from the scope of the present invention, and vice versa.

The terms that are used in the present specification are just intendedto describe particular embodiments, and are not intended to limit thepresent invention. A single element expressed in the specification willbe construed to encompass a plurality of elements unless the contextclearly indicates otherwise. In the present specification, it should beunderstood that the term “include” or “have” is intended to indicate theexistence of characteristics, numbers, steps, operations, elements,parts, or a combination thereof that are described in the specification,and is not intended to exclude the possibility of the existence oraddition of one or more other characteristics, numbers, steps,operations, elements, parts, or a combination thereof.

Unless otherwise defined, all terms including technical and scientificterms, which are used herein, have the same meaning that is commonlyunderstood by those skilled in the art.

The terms that are defined in the general dictionaries shall beconstrued to have the same meaning in the context of the related art,and shall not be construed as an ideal or excessively formal meaningunless clearly defined in the present specification.

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings.

FIG. 1 is a conceptual diagram illustrating an optical tracking system,according to an embodiment of the present invention.

Referring to FIG. 1, the optical tracking system 100, according to anexemplary embodiment of the present invention, includes a marker part110, an image forming part 120, and a processing part 130.

The marker part 110 includes a pattern 112 and a first lens 114.

The pattern 112 has particular information. For example, the particularinformation of the pattern may be recognized by the image forming part120, which will be described later, for tracking, and may includeone-dimensional patterns, such as bar codes, or two-dimensionalpatterns, such as QR codes.

The first lens 114 is spaced apart from the pattern 112, and has a firstfocal length. For example, the distance between the first lens 114 andthe pattern 112 may be the same as the first focal length of the firstlens 114 in order for the image forming part 120, which will bedescribed later, to form an image of the pattern 112 and to track thepattern 112 from a distance. In this case, a bundle of rays with respectto the pattern 112, which pass through the first lens 114, may beparallel. The first lens 114, for example, may perform a similarfunction as an object lens of a microscope.

The marker part 110 may not include a light source. In this case, themarker part 110 may be utilized as a passive marker that uses lightlocated outside of the marker part 110. On the other hand, the markerpart 110 may include a light source. In this case, the marker part 110may be utilized as an active marker that uses its own light.

The image forming part 120 includes a second lens 122 and an imageforming unit 124.

The second lens 122 has a second focal length. The second lens 114, forexample, may perform a similar function as an eyepiece of a microscope.

The image forming unit 124 is spaced apart from the second lens 122 andthe image of the pattern 112 is formed on the image forming unit 124 bythe first lens 114 and the second lens 122. For example, the distancebetween the image forming unit 124 and the second lens 122 may be thesame as the second focal length of the second lens 122 in order to forman image for a bundle of rays with respect to the pattern 112, whichpass through the first lens 114 to be parallel. For example, the imageforming unit 124 may include an image sensor, such as a CCD (chargecoupled device), a CMOS (complementary metal-oxide semiconductor), orthe like.

The processing part 130 determines the posture of the marker part 110from a coordinate conversion formula between the coordinate on thepattern surface of the pattern 112 and a pixel coordinate on the imageof the pattern 112. The processing part 130 tracks the marker part 110by using the determined posture of the marker part 110. The processingpart 130, for example, may include a computer, or more specifically, mayinclude a central processing unit (CPU).

Hereinafter, a system modeling process that becomes the base offunctions of the processing part 130 and a process of determining theposture of the marker part 110 according thereto will be described inmore detail with reference to the drawings.

FIG. 2 is a flowchart schematically showing a problem-solving processthat is necessary for the processing part of the optical tracking systemof FIG. 1 in determining the posture of the marker part.

Referring to FIG. 2, the system modeling is conducted with respect tothe optical tracking system 100, which has the configuration asdescribed above (S100).

In the optical tracking system 100 as shown in FIG. 1, since thecoordinate conversion between the coordinate on the pattern surface ofthe pattern 112 and the pixel coordinate on the image of the pattern 112is made by the optical system of the optical tracking system 100, thecoordinate conversion formula may be configured by modeling thecoordinate conversion according to the optical system of the opticaltracking system 100. At this time, the modeling of the coordinateconversion according to the optical system of the optical trackingsystem 100 may be made by each optical system of the marker part 110 andthe image forming part 120 and by a relationship therebetween.

Then, in the coordinate conversion formula that is acquired as a resultof the system modeling, the first and second conversion matrices, whichwill be described later, are calibrated (S200).

When defining the coordinate on the pattern surface of the pattern 112shown in FIG. 1 as the first coordinate, the three-dimensional localcoordinate of the first coordinate for the first lens 114 as the secondcoordinate, the three-dimensional local coordinate of the secondcoordinate for the second lens 122 as the third coordinate, and thepixel coordinate on the image of the pattern 112 of the image formingpart 120 as the fourth coordinate, the first conversion matrix convertsthe first coordinate to the second coordinate, and the second conversionmatrix converts the third coordinate to the fourth coordinate.

Although the coordinate conversion formula acquired as a result of thesystem modeling is determined as the equation of various parameters ofthe optical systems of the marker part 110 and the image forming part120 shown in FIG. 1, the parameters may not be accurately acquired orvalues thereof may vary with the mechanical arrangement state.Therefore, a more accurate system modeling can be made by calibratingthe first conversion matrix and the second conversion matrix.

Next, a posture definition matrix is acquired by using the calibrationresult (S300).

Here, the posture refers to the direction in which the marker part 110faces, and the posture definition matrix provides information about theposture of the marker part 110 so that roll, pitch, and yaw of themarker part 110 may be recognized from the posture definition matrix.

Hereinafter, each operation of FIG. 2 will be described in more detailwith reference to the drawings.

FIG. 3 is a flowchart illustrating a process of system modeling in theproblem-solving process of FIG. 2, and FIG. 4 is a conceptual diagramfor explaining the process of system modeling in FIG. 3.

Referring to FIGS. 3 and 4, first, equations for three straight linesare acquired according to optical paths between the marker part 110 andthe image forming part 120 (S110).

More specifically, the central point of the first lens 114 is referredto as the first central point A and the central point of the second lens122 is referred to as the second central point O, while point B refersto a certain point on the pattern 112. A ray with respect to a certainpoint B passes straight through the first central point A of the firstlens 114, and the ray that has passed through the first central point Areaches the second lens 122 at point D. Then, the ray is refracted bythe second lens 122 at the point D to then form an image on the imageforming unit 124 at point E. In addition, a ray passes straight throughthe first central point A of the first lens 114 and the second centralpoint O of the second lens 122 to then meet the extension line of theline segment DE at point C.

At this time, the linear equation for the line segment AO (or the linesegment AC), the linear equation for the line segment AD, and the linearequation for the line segment DC are defined as L1, L2, and L3,respectively, as shown in FIG. 4.

In a world coordinate system, the coordinate of the first central pointA is configured as (X,Y,Z), and the coordinate of the second centralpoint O is configured as the origin (0,0,0). Since the coordinate of thesecond central point O of the second lens 122 is configured as theorigin (0,0,0), the three-dimensional local coordinate system for thesecond lens 122 is the same as the world coordinate system.

In addition, the coordinate of a certain point (corresponding to thepoint B) on the pattern 112 is configured as (u,v), and the coordinateof the central point of the pattern 112 is configured as (u_(c),v_(c)).Further, the coordinate of a pixel of the image (corresponding to thepoint E) of the pattern 112, which is formed on the image forming unit124, is configured as (u′,v′). The coordinates (u,v) and (u_(c),v_(c)),for example, may be configured based on the left upper side of pattern112, and the coordinate (u′,v′), for example, may be configured based onthe left upper side of the image of pattern 112.

Meanwhile, when the image forming part 120 is positioned in the focallength f_(c) of the second lens 122, the z-axis coordinate of the imageforming unit 124 may be −f_(c).

The equations of the three straight lines may be acquired in sequence byusing information above.

The equation of the straight line L1 is acquired from the line segmentAO, and the position of the point C is acquired from the same. Theequation of the straight line L2 is acquired from the line segment AB,and the position of the point D is acquired from the same. The equationof the straight line L3 is acquired from the line segment DC. At thistime, since the world coordinates of the point A and the point O aregiven, the equations of the three straight lines may be obtained byrecognizing only the world coordinate of the point B.

When the posture definition matrix for defining the posture of themarker part 110 is defined as a 3*3 matrix [R] and the components of thematrix [R] are defined as r₁₁, r₁₂, r₁₃, r₂₁, r₂₂, r₂₃, r₃₁, r₃₂, andr₃₃, respectively, the world coordinate of the point B may be determinedas (r₁₁u+r₁₂v+r₁₃f_(b)+X, r₂₁u+r₂₂v+r₂₃f_(b)+Y, r₃₁u+r₃₂v+r₃₃f_(b)+Z)that is converted from the pattern coordinate (u,v) of the point B basedon the matrix [R] and the focal length f_(b) of the first lens 114.

Accordingly, it is possible to acquire the equations of the threestraight lines from the world coordinates of the points A, O, and B.

Subsequently, the relational equation between the pattern 112 and theimage of the pattern 112 is induced from the acquired equations of thethree straight lines (S120).

The position of the point E (the world coordinate of the point E) may beacquired from the equation of the straight line L3 obtained above sothat the pixel coordinate (u′,v′) of the point E may be obtained fromthe same.

According to this, since the pixel coordinate (u′,v′) of the point E maybe expressed as the coordinate (u,v) on the pattern of the point B, therelational equation between the pattern 112 and the image of the patterncorresponding to the point E may be determined.

Next, the relational equation is expressed as a matrix equation in orderto thereby configure the same as the coordinate conversion formula(S130).

The relational equation may be expressed as a matrix equation as shownin Equation 1 below, and such a matrix equation for the coordinateconversion may be configured as the coordinate conversion formula.

$\begin{matrix}\begin{matrix}{{s\begin{bmatrix}u_{i}^{\prime} \\v_{i}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A\rbrack\lbrack R\rbrack}\lbrack C\rbrack}\begin{bmatrix}u_{i} \\v_{i} \\1\end{bmatrix}}} \\{= {\begin{bmatrix}{- \frac{f_{c}}{{pw}.}} & 0 & u_{c}^{\prime} \\0 & {- \frac{f_{c}}{{ph}.}} & v_{c}^{\prime} \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}r_{11} & r_{12} & r_{13} \\r_{21} & r_{22} & r_{23} \\r_{31} & r_{32} & r_{33}\end{bmatrix}}} \\{\begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}\begin{bmatrix}u_{i} \\v_{i} \\1\end{bmatrix}}\end{matrix} & ( {{Equation}\mspace{14mu} 1} ) \\( {s = {{r_{31}u_{i}} + {r_{32}v_{i}} + {r_{33}f_{b}}}} ) & \;\end{matrix}$

Here, (u,v) denotes the first coordinate, (u′,v′) denotes the fourthcoordinate, [C] refers to the first conversion matrix, [A] refers to thesecond conversion matrix, [R] refers to the posture definition matrix,(u_(c),v_(c)) denotes the coordinate of the center of the pattern on thepattern surface, f_(b) denotes the first focal length, f_(c) denotes thesecond focal length, pw denotes the width of a pixel of the image of thepattern, ph denotes the height of a pixel of the image of the pattern,and i of (u_(i),v_(i)) and (u′_(i), v′_(i)) indicates the predeterminedi-th pattern.

It can be seen that the coordinate conversion formula is made by theproduct of the first and second conversion matrices, which are describedin FIG. 1, and the posture definition matrix.

More specifically, as described in FIG. 1, when defining the coordinateon the pattern surface of the pattern 112 as the first coordinate (u,v),the three-dimensional local coordinate of the first coordinate for thefirst lens 114 as the second coordinate, the three-dimensional localcoordinate (equal to the world coordinate) of the second coordinate forthe second lens 122 as the third coordinate, and the pixel coordinate onthe image of the pattern 112 of the image forming part 120 as the fourthcoordinate (u′,v′), it can be seen that the coordinate conversionformula is conceptually expressed as [A][R][C], which is the product ofthe first conversion matrix [C] for converting the first coordinate tothe second coordinate, the posture definition matrix [R] for convertingthe second coordinate to the third coordinate, and the second conversionmatrix [A] for converting the third coordinate to the fourth coordinate.

Next, the operation S200 of calibrating the first and second conversionmatrices in the coordinate conversion formula that is acquired as aresult of the system modeling will be described in more detail withreference to the drawings.

The calibration is carried out first with respect to the secondconversion matrix, and is then carried out with respect to the firstconversion matrix.

FIG. 5 is a flowchart illustrating a process of calibrating the secondconversion matrix in the problem-solving process of FIG. 2.

Referring to FIG. 5, first, a matrix [B] and a matrix [H] are defined tofacilitate the mathematical analysis for the calibration (S210).

More specifically, the matrix [B] may be defined by using the secondconversion matrix [A] as shown in Equation 2, and the matrix [H] may bedefined by using the first conversion matrix [C], the second conversionmatrix [A], and the posture definition matrix [R] as shown in Equation3.[B]=[A] ^(−T) *[A] ⁻¹  (Equation 2)[H]=[A][R][C]  (Equation 3)

Here, all of the matrixes [A], [B], [C], [H], and [R] are in the form ofa 3*3 matrix, and it may be expressed that [H]=[h1,h2,h3] and[R]=[r1,r2,r3].

Equation 4 is obtained by multiplying both sides of Equation 3 by A⁻¹.A ⁻¹ [h ₁ h ₂ h ₃ ]=[r ₁ r ₂ T]  (Equation 4)

Then, the equation comprised of the components of [H] and [B] isconfigured by using the orthonormality of the matrix [R] (S220).

More specifically, the matrix [B] may be defined as shown in Equation 5by using the orthonormality of the posture definition matrix [R]corresponding to a rotation matrix.

$\begin{matrix}{\lbrack B\rbrack = {\begin{bmatrix}\frac{1}{\alpha^{2}} & 0 & {- \frac{u_{c}^{\prime}}{\alpha^{2}}} \\0 & \frac{1}{\beta^{2}} & {- \frac{v_{c}^{\prime}}{\beta^{2}}} \\{- \frac{u_{c}^{\prime}}{\alpha^{2}}} & {- \frac{v_{c}^{\prime}}{\beta^{2}}} & {\frac{u_{c}^{\prime^{2}}}{\alpha^{2}} + \frac{v_{c}^{\prime^{2}}}{\beta^{2}} + 1}\end{bmatrix} = \begin{bmatrix}B_{11} & 0 & B_{13} \\0 & B_{22} & B_{23} \\B_{13} & B_{23} & B_{33}\end{bmatrix}}} & ( {{Equation}\mspace{11mu} 5} )\end{matrix}$

Here, α=−f_(c)/pw, β=−f_(c)/ph, f_(c) refers to the focal length of thesecond lens 122 of the image forming part 120, and Pw and ph refer tothe width and the height of a pixel, respectively.

Column vectors b and v_(ij) are defined as shown in Equation 6 by usingnon-zero components of the matrix [B].b=[B ₁₁ B ₂₂ B ₁₃ B ₂₃ B ₃₃]^(T)v _(ij) =[h _(i1) h _(j1) ,h _(i2) h _(j2) ,h _(i3) h _(j1) +h _(i1) h_(j3) ,h _(i3) h _(j2) +h _(i2) h _(j3) ,h _(i3) h _(j3)]^(T)  (Equation 6)

Equation 7 may be acquired by using the orthonormality of the matrix [R]with respect to Equation 6.

$\begin{matrix}{\mspace{79mu}{{\begin{matrix}\begin{bmatrix}v_{12}^{T} \\( {v_{11} - v_{22}} )^{T}\end{bmatrix} & {b = 0}\end{matrix}\begin{bmatrix}{{h_{11}h_{21}},{h_{12}h_{22}},{{h_{13}h_{21}} + {h_{11}h_{23}}},{{h_{13}h_{22}} + {h_{12}h_{23}}},{h_{13}h_{23}}} \\{{{h_{11}h_{11}} - {h_{21}h_{21}}},{{h_{12}h_{12}} - {h_{22}h_{22}}},{{h_{13}h_{11}} + {h_{11}h_{13}} -}} \\{( {{h_{23}h_{21}} + {h_{21}h_{23}}} ),{{h_{13}h_{12}} + {h_{12}h_{13}} - ( {{h_{23}h_{22}} + {h_{22}h_{23}}} )},} \\{{h_{13}h_{13}} - {h_{23}h_{23}}}\end{bmatrix}}{\quad{\lbrack \begin{matrix}B_{11} \\B_{22} \\B_{13} \\B_{23} \\B_{33}\end{matrix} \rbrack = 0}}}} & ( {{Equation}\mspace{14mu} 7} )\end{matrix}$

Next, values of the matrix [B] are obtained by applying data on three ormore images to the matrix [H] (S230).

More specifically, after applying three or more images to Equation 7,the column vector b may be obtained by using a method such as singularvalue decomposition (SVD). Once the column vector b is obtained, allcomponents of the matrix [B] may be recognized.

Next, the calibrated matrix [A] is eventually acquired (S240).

More specifically, when all components of the matrix [B] are given,v′_(c), α, β, and u′_(c) may be obtained through Equation 8 (λ and γ areexpressed as parameters).

$\begin{matrix}{{v_{c}^{\prime} = \frac{{B_{12}B_{13}} - {B_{11}B_{23}}}{{B_{11}B_{22}} - B_{12}^{2}}}{{\alpha = \sqrt{\frac{\lambda}{B_{11}}}},\mspace{14mu}{\beta = \sqrt{\frac{\lambda\; B_{11}}{{B_{11}B_{22}} - B_{12}^{2}}}}}{u_{c}^{\prime} = {\frac{\gamma\; v_{c}^{\prime}}{\beta} - \frac{B_{13}\alpha^{2}}{\lambda}}}{\lambda = {B_{33} - \frac{\lbrack {B_{13}^{2} + {v_{c}^{\prime}( {{B_{12}B_{13}} - {B_{11}B_{23}}} )}} \rbrack}{B_{11}}}}{\gamma = {{- B_{12}}\alpha^{2}{\beta/\lambda}}}} & ( {{Equation}\mspace{14mu} 8} )\end{matrix}$

Therefore, all components of the matrix [A] may be obtained fromEquation 9.

$\begin{matrix}{{\lbrack A\rbrack = \begin{bmatrix}\alpha & 0 & u_{c}^{\prime} \\0 & \beta & v_{c}^{\prime} \\0 & 0 & 1\end{bmatrix}};( {{\alpha = {- \frac{f_{c}}{{pw}.}}},{\beta = {- \frac{f_{c}}{{ph}.}}}} )} & ( {{Equation}\mspace{14mu} 9} )\end{matrix}$

Next, the calibration of the first conversion matrix [C] is made byusing the second conversion matrix [A] that has been previouslycalibrated.

FIG. 6 is a flowchart illustrating a process of calibrating the firstconversion matrix in the problem-solving process of FIG. 2.

Referring to FIG. 6, the matrix [R] is obtained by putting thecalibrated matrix [A] in the equation on matrix [H] (S250).

More specifically, Equation 10 is acquired by putting the secondconversion matrix [A] of Equation 9 in Equation 3 and by calculating[R][C] of Equation 1.

$\begin{matrix}\begin{matrix}{\lbrack H\rbrack = {{\lbrack A\rbrack\lbrack R\rbrack}\lbrack C\rbrack}} \\{= {\lbrack A\rbrack\lbrack{RC}\rbrack}} \\{= {\begin{bmatrix}\alpha & 0 & u_{c}^{\prime} \\0 & \beta & v_{c}^{\prime} \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}r_{11} & r_{12} & {{{- u_{c}}r_{11}} - v_{c} - {u_{c}r_{12}} + {f_{b}r_{13}}} \\r_{21} & r_{22} & {{{- u_{c}}r_{21}} - v_{c} - {u_{c}r_{22}} + {f_{b}r_{23}}} \\r_{31} & r_{32} & {{{- u_{c}}r_{31}} - v_{c} - {u_{c}r_{32}} + {f_{b}r_{33}}}\end{bmatrix}}} \\{= \begin{bmatrix}h_{1} & h_{2} & h_{3}\end{bmatrix}}\end{matrix} & ( {{Equation}\mspace{14mu} 10} )\end{matrix}$

If the matrix [R] is replaced by [R]=[r1 r2 r3] in Equation 10, [R] maybe obtained for each column vector component from Equation 11.r ₁ [A] ⁻¹ h ₁ ,r ₂ =[A] ⁻¹ h ₂ ,r ₃ =r ₁ ×r ₂  (Equation 11)

Subsequently, a matrix [HK] is defined as [HK]=[A][R] to then be appliedto the coordinate conversion formula (S260).

More specifically, the product of the matrix [A] and the matrix [R] isdefined as the matrix [HK] to then be applied to the coordinateconversion formula of Equation 1 to have the components of the matrix[HK] and the matrix [C].

At this time, the matrix [HK] may be obtained by using the matrix [A]that is acquired in Equation 9 and the matrix [R] that is acquired inEquation 11, and may be applied to the coordinate conversion formula ofEquation 1 in order to thereby acquire Equation 12, which comprises ofthe components of the matrix [HK] and the matrix [C].

$\begin{matrix}{{s\begin{bmatrix}u^{\prime} \\v^{\prime} \\1\end{bmatrix}} = {{{{\lbrack A\rbrack\lbrack R\rbrack}\lbrack C\rbrack}\begin{bmatrix}u \\v \\1\end{bmatrix}} = {\quad{{{\lbrack{HK}\rbrack\lbrack C\rbrack}\begin{bmatrix}u \\v \\1\end{bmatrix}} = {{\lbrack{HK}\rbrack\begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}}\begin{bmatrix}u \\v \\1\end{bmatrix}}}}}} & ( {{Equation}\mspace{14mu} 12} )\end{matrix}$

Next, the resultant equation is transformed into a form of [AA][CC]=[BB](S270).

More specifically, the matrix comprised of only the components of thematrix [C] is defined as [CC] and is separated from the resultantequation in order to thereby transform the same into a form of[AA][CC]=[BB]. At this time, since the matrix [HK] is known, the matrix[AA], the matrix [BB], and the matrix [CC] may be defined as shown inEquation 13 by using the matrix [HK].

$\mspace{985mu}{{( {{Equation}\mspace{14mu} 13} )\lbrack{AA}\rbrack} = {{\begin{bmatrix}{{{HK}( {2,1} )} - {{{HK}( {3,1} )}v_{i}^{\prime}}} & {{{HK}( {2,2} )} - {{{HK}( {3,2} )}v_{i}^{\prime}}} & {{{HK}( {2,3} )} + {{{HK}( {3,3} )}v_{i}^{\prime}}} \\{{- {{HK}( {1,1} )}} + {{{HK}( {3,1} )}u_{i}^{\prime}}} & {{- {{HK}( {1,2} )}} + {{{HK}( {3,2} )}u_{i}^{\prime}}} & {{- {{HK}( {1,3} )}} - {{{HK}( {3,3} )}u_{i}^{\prime}}}\end{bmatrix}\lbrack{BB}\rbrack} = {{\begin{bmatrix}{{{{HK}( {2,1} )}u_{i}} + {{{HK}( {2,2} )}v_{i}} - {{{HK}( {3,1} )}v_{i}^{\prime}u_{i}} - {{{HK}( {3,2} )}v_{i}^{\prime}v_{i}}} \\{{{- {{HK}( {1,1} )}}u_{i}} - {{{HK}( {1,2} )}v_{i}} + {{{HK}( {3,1} )}u_{i}^{\prime}u_{i}} + {{{HK}( {3,2} )}u_{i}^{\prime}v_{i}}}\end{bmatrix}\lbrack{CC}\rbrack} = \begin{bmatrix}u_{c} \\v_{c} \\f_{b}\end{bmatrix}}}}$

Then, [CC] is obtained from [CC]=[AA]⁻¹[BB] in order to thereby acquirethe matrix [C] that is calibrated (S280).

More specifically, the components of [CC] are acquired from[CC]=[AA]⁻¹[BB] that is transformed from the equation [AA][CC]=[BB] tofinally obtain the first conversion matrix [C] that is calibrated.

Next, the operation S300 of acquiring the posture definition matrix byusing the first and second conversion matrices, which have beencalibrated, will be described in more detail with reference to thedrawings.

FIG. 7 is a flowchart illustrating an example of a process for acquiringa posture definition matrix in problem-solving process of FIG. 2.

Referring to FIG. 7, as an example for obtaining the posture definitionmatrix [R], first, an equation, which is obtained from the vectorproduct by itself with respect to both sides thereof, is configured(S310).

More specifically, since the vector product of Equation 1 by itself onboth sides thereof results in zero, Equation 14 may be acquired byconfiguring the same as an equation.

$\begin{matrix}{{\begin{bmatrix}u_{i}^{\prime} \\v_{i}^{\prime} \\1\end{bmatrix} \times \begin{bmatrix}u_{i}^{\prime} \\v_{i}^{\prime} \\1\end{bmatrix}} = {{{{{\begin{bmatrix}0 & {- 1} & v_{i}^{\prime} \\1 & 0 & {- u_{i}^{\prime}} \\{- v_{i}^{\prime}} & u_{i}^{\prime} & 0\end{bmatrix}\lbrack A\rbrack}\lbrack R\rbrack}\lbrack C\rbrack}\begin{bmatrix}u_{i} \\v_{i} \\1\end{bmatrix}} = 0}} & ( {{Equation}\mspace{14mu} 14} )\end{matrix}$

Next, the matrix [H] is acquired by using such a method as singularvalue decomposition (SVD) (S320 a).

More specifically, [H]=[A][R][C] of Equation 3 is applied to Equation 14to then make an equation with respect to the components (H1, H2, . . . ,H9) of the matrix [H] in order to thereby obtain Equation 15.

$\mspace{565mu}{{{( {{Equation}\mspace{14mu} 15} )\begin{bmatrix}0 & 0 & 0 & u_{1} & v_{1} & {- 1} & {u_{1}v_{1}^{\prime}} & {v_{1}v_{1}^{\prime}} & v_{1}^{\prime} \\u_{1} & v_{1} & 1 & 0 & 0 & 0 & {u_{1}u_{1}^{\prime}} & {v_{1}u_{1}^{\prime}} & u_{1}^{\prime} \\\vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\0 & 0 & 0 & u_{n} & v_{n} & {- 1} & {u_{n}v_{n}^{\prime}} & {v_{n}v_{n}^{\prime}} & v_{n}^{\prime} \\u_{n} & v_{n} & 1 & 0 & 0 & 0 & {u_{n}u_{n}^{\prime}} & {v_{n}u_{n}^{\prime}} & u_{n}^{\prime}\end{bmatrix}}\begin{bmatrix}H_{1} \\H_{2} \\H_{3} \\H_{4} \\H_{5} \\H_{6} \\H_{7} \\H_{8} \\H_{9}\end{bmatrix}} = 0}$

Using a method such as singular value decomposition (SVD), 2n equationsof Equation 15 are acquired.

Next, [R] is obtained from [R]=[A]⁻¹[H][C]⁻¹ (S330 a).

More specifically, [R] is obtained from [R]=[A]⁻¹[H][C]⁻¹ that istransformed from [H]=[A][R][C] of Equation 3.

The posture definition matrix may be obtained by other methods.

FIG. 8 is a flowchart illustrating another example of a process foracquiring a posture definition matrix in problem-solving process of FIG.2.

Referring to FIG. 8, as another example for obtaining the posturedefinition matrix [R], first, an equation, which is obtained from thevector product by itself with respect to both sides thereof (S310), isconfigured. This operation is the same as that of FIG. 7, so thatduplicate description thereof will be omitted.

Subsequently, the equation is rewritten as an equation for r₁₁˜r₃₃ (S320b).

More specifically, the equation with respect to each component r₁₁, r₁₂,r₁₃, r₂₁, r₂₂, r₂₃, r₃₁, r₃₂, or r₃₃ of the posture definition matrix[R] is made from Equation 14 in order to thereby acquire Equation 16.

${{{( {{Equation}\mspace{14mu} 16} )\begin{bmatrix}{\frac{f_{c}}{pw}u_{1}} & {\frac{f_{c}}{pw}v_{1}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {u_{1}^{\prime} - u_{c}^{\prime}} )u_{1}} & {( {u_{1}^{\prime} - u_{c}^{\prime}} )v_{1}} & {( {u_{1}^{\prime} - u_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}u_{1}} & {\frac{f_{c}}{ph}v_{1}} & {\frac{f_{c}}{ph}f_{b}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )u_{1}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )v_{1}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )f_{b}} \\\; & \; & \; & \; & \; & \vdots & \; & \; & \; \\{\frac{f_{c}}{pw}u_{n}} & {\frac{f_{c}}{pw}v_{n}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {u_{n}^{\prime} - u_{c}^{\prime}} )u_{n}} & {( {u_{n}^{\prime} - u_{c}^{\prime}} )v_{n}} & {( {u_{n}^{\prime} - u_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}u_{n}} & {\frac{f_{c}}{ph}v_{n}} & {\frac{f_{c}}{ph}f_{b}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )u_{n}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )v_{n}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )f_{b}}\end{bmatrix}}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}$

Next, the matrix [R] is obtained by using a method such as singularvalue decomposition (SVD) (S330 b).

More specifically, 2n equations of Equation 16 are acquired by using amethod such as singular value decomposition (SVD).

As described above, [R] is finally obtained.

The posture of the marker part 110 may be calculated by applying thesystem modeling process and the method for acquiring the posturedefinition matrix [R] described above to the optical tracking system 100shown in FIG. 1.

Hereinafter, the method of calculating the posture of the marker part110 by the processing part 130 will be described in more detail withreference to the drawings.

FIG. 9 is a flowchart illustrating a method of calculating the postureof the marker part of the optical tracking system, according to anembodiment of the present invention.

Referring to FIG. 9, first, the processing part 130 calibrates the firstand second conversion matrices from three or more images (S510).

The calibration may be substantially the same as operation S200described in FIG. 2 and operations S210 to S280 described in detail inFIGS. 5 and 6. The processing part 130 may calibrate the first andsecond conversion matrices by using only the final equation for thecalibration as in operations S230 and S280 among the operations above.

Next, the posture definition matrix is acquired from the coordinateconversion formula that contains the first and second conversionmatrices (S520).

The acquisition of the posture definition matrix may be substantiallythe same as operation S300 described in FIG. 2, operations S310 to S330a, and operations S310 to S330 b described in detail in FIGS. 7 and 8.The processing part 130 may acquire the posture definition matrix byusing only the final equation for the acquisition of the posturedefinition matrix as in operation S320 a and S320 b among the operationsabove.

Accordingly, the processing part 130 may acquire the first conversionmatrix for converting the first coordinate to the second coordinate andthe second conversion matrix for converting the third coordinate to thefourth coordinate through the calibration in advance, and may acquirethe posture definition matrix for defining the posture of the markerpart 110 from the coordinate conversion formula.

Once the posture definition matrix is acquired, the posture of themarker part 110 may be recognized. For example, the roll, pitch, and yawof the marker part 110 may be recognized from the posture definitionmatrix.

According to the optical tracking system described above, the markerpart can be miniaturized while including a pattern of particularinformation to enable the tracking, and the posture of the marker partcan be determined by modeling the optical system of the marker part andthe image forming part with the coordinate conversion formula.Therefore, it is possible to accurately track the marker part by asimpler and easier method.

FIG. 10 is a conceptual diagram illustrating an optical tracking system,according to another embodiment of the present invention.

Referring to FIG. 10, the optical tracking system 1000, according toanother embodiment of the present invention, includes a marker part1110, a first image forming part 1120 a, a second image forming part1120 b, and a processing part 1130.

The optical tracking system 1000 illustrated in FIG. 10 is substantiallyidentical to the optical tracking system 100 illustrated in FIG. 1,except that a stereo structure including two image forming parts isapplied and the processing part 1130 determines the posture by using twoimage forming parts, so the duplicate description thereof will beomitted.

The marker part 1110 includes a pattern 1112 and a first lens 1114, andis substantially the same as the marker part 110 illustrated in FIG. 1.

The first image forming part 1120 a includes: a second lens that has asecond focal length; and a first image forming unit that is spaced apartfrom the second lens and on which the first image of the pattern isformed by the first lens and the second lens.

The second image forming part 1120 b includes: a third lens that has athird focal length; and a second image forming unit that is spaced apartfrom the third lens and on which the second image of the pattern isformed by the first lens and the third lens.

Each of the first image forming part 1120 a and the second image formingpart 1120 b is substantially the same as the image forming part 120shown in FIG. 1.

The processing part 1130 determines the posture of the marker part 1110from: the first coordinate conversion formula between the coordinate onthe pattern surface of the pattern 1112 and the first pixel coordinateon the first image of the pattern 1112; and the second coordinateconversion formula between the coordinate on the pattern surface of thepattern 1112 and the second pixel coordinate on the second image of thepattern 1112. At this time, the second coordinate conversion formulaincludes a rotation conversion between the first pixel coordinate andthe second pixel coordinate on the second image. The processing part1130 tracks the marker part 1110 by using the determined posture of themarker part 1110.

Hereinafter, a system modeling process that is the base of functions ofthe processing part 1130 and a process of determining the posture of themarker part 1110 according thereto will be described in more detail withreference to the drawings.

FIG. 11 is a flowchart schematically showing a problem-solving processthat is necessary for the processing part of the optical tracking systemof FIG. 10 to determine the posture of the marker part.

Referring to FIG. 11, first, the system modeling is conducted withrespect to the optical tracking system 1000 that has the configurationas described above (S1100).

In the optical tracking system 1000 as shown in FIG. 10, since thecoordinate conversion between the coordinate on the pattern surface ofthe pattern 1112 and the pixel coordinate on the image of the pattern1112 is made by the optical system of the optical tracking system 1000,the coordinate conversion formula may be configured by modeling thecoordinate conversion according to the optical system of the opticaltracking system 1000. At this time, the modeling of the coordinateconversion according to the optical system of the optical trackingsystem 1000 may be made by each optical system of the marker part 1110and the first and second image forming parts 1120 a and 1120 b and by arelationship therebetween.

Then, among the coordinate conversion formula that is acquired as aresult of the system modeling, the first, second, third, and fourthconversion matrixes, which will be described later, are calibrated(S1200).

When defining: the coordinate on the pattern surface of the pattern 1112shown in FIG. 10 as the first coordinate; the three-dimensional localcoordinate of the first coordinate for the first lens 1114 as the secondcoordinate; the three-dimensional local coordinate of the secondcoordinate for the second lens 1122 a as the third coordinate; and thepixel coordinate on the first image of the pattern 1112 of the firstimage forming part 1120 a as the fourth coordinate, respectively, thefirst conversion matrix converts the first coordinate to the secondcoordinate, and the second conversion matrix converts the thirdcoordinate to the fourth coordinate.

In addition, when defining: the coordinate on the pattern surface of thepattern 1112 shown in FIG. 10 as the fifth coordinate; thethree-dimensional local coordinate of the fifth coordinate for the firstlens 1114 as the sixth coordinate; the three-dimensional localcoordinate of the sixth coordinate for the third lens 1122 b as theseventh coordinate; and the pixel coordinate on the second image of thepattern 1112 of the second image forming part 1120 b as the eighthcoordinate, respectively, the third conversion matrix converts the fifthcoordinate to the sixth coordinate, and the fourth conversion matrixconverts the seventh coordinate to the eighth coordinate.

Although the coordinate conversion formula that is acquired as a resultof the system modeling is determined as an equation of variousparameters of the optical systems of the marker part 1110 and the imageforming part 1120 shown in FIG. 10, the parameters may not be accuratelyacquired or values thereof may vary with the mechanical arrangementstate. Therefore, the more accurate system modeling can be made bycalibrating the first conversion matrix and the second conversionmatrix.

Next, a posture definition matrix is acquired by using the calibrationresult (S1300).

The posture definition matrix provides information about the posture ofthe marker part 1110 so that the roll, pitch, and yaw of the marker part1110 may be recognized from the posture definition matrix.

Hereinafter, each operation of FIG. 11 will be described in more detail.

First, the method and the result, which have been described in FIG. 3,are equally applied to the system modeling (S1100). Therefore, Equation1 may be independently applied to the first image forming part 1120 aand/or the second image forming part 1120 b. According to this, Equation17 below may be acquired as a result of the system modeling. For theconvenience, in Equation 17, L and l are applied to the variables forthe first image forming part 1120 a, and R and r are applied to thevariables for the second image forming part 1120 b.

$\begin{matrix}{\mspace{79mu}{{{s\begin{bmatrix}{lu}^{\prime} \\{lv}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A_{l} \rbrack\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{lu} \\{lv} \\1\end{bmatrix}}}{{s\begin{bmatrix}{ru}^{\prime} \\{rv}^{\prime} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{R} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{LR} \rbrack}\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}}}}}} & ( {{Equation}\mspace{14mu} 17} )\end{matrix}$

Here, (lu,lv) denotes the first coordinate. (lu′,lv′) denotes the fourthcoordinate. [C] denotes the first conversion matrix. [Al] denotes thesecond conversion matrix. [RL] denotes the first posture definitionmatrix. (ru,rv) denotes the fifth coordinate. (ru′,rv′) denotes theeighth coordinate. [C] denotes the third conversion matrix that is thesame as the first conversion matrix. [Ar] denotes the fourth conversionmatrix. In addition, [RR] denotes the second posture definition matrix.In addition, referring to FIG. 10, the matrix [RR] may be expressed as amatrix [RLR][RL]. Here, [RLR] indicates the matrix that converts thethree-dimensional local coordinate for the first image forming part 1120a to the three-dimensional local coordinate for the second image formingpart 1120 b, and may be a matrix for defining the posture of the firstimage forming part 1120 a with respect to the second image forming part1120 b.

Next, the operation S1200 of calibrating the first, second, third, andfourth conversion matrixes in the coordinate conversion formula that isacquired as a result of the system modeling will be described in moredetail with reference to the drawings.

FIG. 12 is a flowchart illustrating an operation of calibratingconversion matrixes in the problem-solving process of FIG. 11.

The operation of calibrating conversion matrixes is basically the sameas the operation that is described in FIGS. 5 and 6 above.

Referring to FIG. 12, first, [Al] is calibrated by applying operationsS210 to S240 to the first image forming part 1120 a (S1210 a). Next,[RL] is acquired by applying operation S250 to the first image formingpart 1120 a (S1220 a).

In addition, in parallel with this, [Ar] is calibrated by applyingoperations S210 to S240 to the second image forming part 1120 b (S1210b). Next, [RR] is acquired by applying operation S250 to the secondimage forming part 1120 b (S1220 b).

As described above, [C] is calibrated in a similar manner as describedin FIGS. 5 and 6 by using [Al], [RL], [Ar], and [RR] that areindependently acquired according to the method described in FIGS. 5 and6.

More specifically, first, the matrix [HKl] is defined as [HKl]=[Al][RL]and the [HKr] is defined as [HKr]=[Ar] [RR] to then be applied to eachcoordinate conversion formula (S1260). This operation corresponds tooperation S260 that is described in FIG. 6, and thus, Equation 18 isacquired, which is comprised of the components of the matrixes [HKl],the matrix [HKr], and the matrix [C].

$\begin{matrix}{{s_{l}\begin{bmatrix}{lu}_{i}^{\prime} \\{lv}_{i}^{\prime} \\1\end{bmatrix}} = {{{{\lbrack A_{l} \rbrack\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{lu}_{i} \\{lv}_{i} \\1\end{bmatrix}} = {\quad{{{\lbrack {HK}_{l} \rbrack\lbrack C\rbrack}\begin{bmatrix}{lu}_{i} \\{lv}_{i} \\1\end{bmatrix}} = {{{\lbrack {HK}_{l} \rbrack\begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}}\begin{bmatrix}{lu}_{i} \\{lv}_{i} \\1\end{bmatrix}} = {{0{s_{r}\begin{bmatrix}{ru}_{i}^{\prime} \\{rv}_{i}^{\prime} \\1\end{bmatrix}}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{R} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru}_{i} \\{rv}_{i} \\1\end{bmatrix}} = {{{\lbrack {HK}_{r} \rbrack\lbrack C\rbrack}\begin{bmatrix}{ru}_{i} \\{rv}_{i} \\1\end{bmatrix}} = {\quad{{{\lbrack {HK}_{r} \rbrack\begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}}\begin{bmatrix}{ru}_{i} \\{rv}_{i} \\1\end{bmatrix}} = 0}}}}}}}}}} & ( {{Equation}\mspace{14mu} 18} )\end{matrix}$

Next, the obtained equation is transformed into a form of[AA2][CC2]=[BB2] (S1270). This operation corresponds to operation S270that is described in FIG. 6, and at this time, the matrix [AA2], thematrix [BB2], and the matrix [CC2] may be defined as Equation 19.

${{( {{Equation}\mspace{14mu} 19} )\lbrack {{AA}\; 2} \rbrack} = {{\begin{bmatrix}{{{HK}_{l}( {2,1} )} - {{{HK}_{l}( {3,1} )}{lv}_{i}^{\prime}}} & {{{HK}_{l}( {2,2} )} - {{{HK}_{l}( {3,2} )}{lv}_{i}^{\prime}}} & {{- {{HK}_{l}( {2,3} )}} + {{{HK}_{l}( {3,3} )}{lv}_{i}^{\prime}}} \\{{- {{HK}_{l}( {1,1} )}} + {{{HK}_{l}( {3,1} )}{lu}_{i}^{\prime}}} & {{- {{HK}_{l}( {1,2} )}} + {{{HK}_{l}( {3,2} )}{lu}_{i}^{\prime}}} & {{{HK}_{l}( {1,3} )} - {{{HK}_{l}( {3,3} )}{lu}_{i}^{\prime}}} \\{{{HK}_{r}( {2,1} )} - {{{HK}_{r}( {3,1} )}{rv}_{i}^{\prime}}} & {{{HK}_{r}( {2,2} )} - {{{HK}_{r}( {3,2} )}{rv}_{i}^{\prime}}} & {{- {{HK}_{r}( {2,3} )}} + {{{HK}_{r}( {3,3} )}{rv}_{i}^{\prime}}} \\{{- {{HK}_{r}( {1,1} )}} + {{{HK}_{r}( {3,1} )}{ru}_{i}^{\prime}}} & {{- {{HK}_{r}( {1,2} )}} + {{{HK}_{r}( {3,2} )}{ru}_{i}^{\prime}}} & {{{HK}_{r}( {1,3} )} - {{{HK}_{r}( {3,3} )}{ru}_{i}^{\prime}}}\end{bmatrix}\lbrack {{BB}\; 2} \rbrack} = {{\begin{bmatrix}{{{{HK}_{l}( {2,1} )}{lu}_{i}} + {{{HK}_{l}( {2,2} )}{lv}_{i}} - {{{HK}_{l}( {3,1} )}{lv}_{i}^{\prime}{lu}_{i}} - {{{HK}_{l}( {3,2} )}{lv}_{i}^{\prime}{lv}_{i}}} \\{{{- {{HK}_{l}( {1,1} )}}{lu}_{i}} - {{{HK}_{l}( {1,2} )}{lv}_{i}} + {{{HK}_{l}( {3,1} )}{lu}_{i}^{\prime}{lu}_{i}} + {{{HK}_{l}( {3,2} )}{lu}_{i}^{\prime}{lv}_{i}}} \\{{{{HK}_{r}( {2,1} )}{ru}_{i}} + {{{HK}_{r}( {2,2} )}{rv}_{i}} - {{{HK}_{r}( {3,1} )}{rv}_{i}^{\prime}{ru}_{i}} - {{{HK}_{r}( {3,2} )}{rv}_{i}^{\prime}{rv}_{i}}} \\{{{- {{HK}_{r}( {1,1} )}}{ru}_{i}} - {{{HK}_{r}( {1,2} )}{rv}_{i}} + {{{HK}_{r}( {3,1} )}{ru}_{i}^{\prime}{ru}_{i}} + {{{HK}_{r}( {3,2} )}{ru}_{i}^{\prime}{rv}_{i}}}\end{bmatrix}\lbrack {{CC}\; 2} \rbrack} = \begin{bmatrix}u_{c} \\v_{c} \\f_{b}\end{bmatrix}}}}$

Next, [CC2] is obtained from [CC2]=[AA2]−1[BB2] in order to therebyacquire the calibrated [C] (S1280). This operation corresponds tooperation S280 that is described in FIG. 6, and the components of [CC2]from [CC2]=[AA2]−1[BB2] that is transformed from [AA2][CC2]=[BB2] tothen eventually acquire [C] corresponding to the first and thirdcalibrated conversion matrixes.

Next, the operation S1300 of acquiring the posture definition matrix byusing the first, second, third, and fourth calibrated conversionmatrixes will be described in more detail with reference to thedrawings.

FIG. 13 is a flowchart illustrating an example of a process of acquiringa posture definition matrix in the problem-solving process of FIG. 11.

The operation of acquiring the posture definition matrix is basicallythe same as the operation described in FIG. 8 above, except that it usesmore data as a stereo structure.

Referring to FIG. 13, first, operations S310 and S320 b are applied to[RL] in the first coordinate conversion formula in order to thereby makethe first equation for the components r11˜r33 (S1310). This operationcorresponds to operations S310 and S320 b described in FIG. 8, and thefirst equation is shown in Equation 20.

${{( {{Equation}\mspace{14mu} 20} )\lbrack R_{L} \rbrack} = {{{\begin{bmatrix}r_{11} & r_{12} & r_{13} \\r_{21} & r_{22} & r_{23} \\r_{31} & r_{32} & r_{33}\end{bmatrix}\begin{bmatrix}{\frac{f_{c}}{pw}u_{1}} & {\frac{f_{c}}{pw}v_{1}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {u_{1}^{\prime} - u_{c}^{\prime}} )u_{1}} & {( {u_{1}^{\prime} - u_{c}^{\prime}} )v_{1}} & {( {u_{1}^{\prime} - u_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}u_{1}} & {\frac{f_{c}}{ph}v_{1}} & {\frac{f_{c}}{ph}f_{b}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )u_{1}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )v_{1}} & {( {v_{1}^{\prime} - v_{c}^{\prime}} )f_{b}} \\\; & \; & \; & \; & \; & \vdots & \; & \; & \; \\{\frac{f_{c}}{pw}u_{n}} & {\frac{f_{c}}{pw}v_{n}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {u_{n}^{\prime} - u_{c}^{\prime}} )u_{n}} & {( {u_{n}^{\prime} - u_{c}^{\prime}} )v_{n}} & {( {u_{n}^{\prime} - u_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}u_{n}} & {\frac{f_{c}}{ph}v_{n}} & {\frac{f_{c}}{ph}f_{b}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )u_{n}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )v_{n}} & {( {v_{n}^{\prime} - v_{c}^{\prime}} )f_{b}}\end{bmatrix}}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}}$

Subsequently, the components of [R_(LR)] are substituted by r′₁₁˜r′₃₃ inthe second coordinate conversion formula, and operations S310 and S320 bare applied to [R_(L)] in order to make the second equation for r₁₁˜r₃₃(S1320). This operation is the application of operations S310 and S320 bdescribed in FIG. 8, and the second equation is shown in Equation 21below.

$\begin{matrix}{{\lbrack R_{LR} \rbrack = {{{\begin{bmatrix}r_{11}^{\prime} & r_{12}^{\prime} & r_{13}^{\prime} \\r_{21}^{\prime} & r_{22}^{\prime} & r_{23}^{\prime} \\r_{31}^{\prime} & r_{32}^{\prime} & r_{33}^{\prime}\end{bmatrix}\lbrack \begin{matrix}{{A\; 1\; r_{11}^{\prime}} + {B\; 1\; r_{31}^{\prime}}} & {{C\; 1\; r_{11}^{\prime}} + {D\; 1\; r_{31}^{\prime}}} & {{E\; 1\; r_{11}^{\prime}} + {F\; 1\; r_{31}^{\prime}}} & {{A\; 1\; r_{12}^{\prime}} + {B\; 1\; r_{32}^{\prime}}} & {{C\; 1\; r_{12}^{\prime}} + {D\; 1\; r_{302}^{\prime}}} & {{E\; 1\; r_{12}^{\prime}} + {F\; 1\; r_{32}^{\prime}}} & {{A\; 1\; r_{13}^{\prime}} + {B\; 1\; r_{33}^{\prime}}} & {{C\; 1\; r_{13}^{\prime}} + {D\; 1\; r_{33}^{\prime}}} & {{E\; 1\; r_{13}^{\prime}} + {F\; 1\; r_{33}^{\prime}}} \\{{A\; 2\; r_{11}^{\prime}} + {B\; 2\; r_{31}^{\prime}}} & {{C\; 12r_{11}^{\prime}} + {D\; 2\; r_{31}^{\prime}}} & {{E\; 2\; r_{11}^{\prime}} + {F\; 2\; r_{31}^{\prime}}} & {{A\; 2\; r_{12}^{\prime}} + {B\; 2\; r_{32}^{\prime}}} & {{C\; 2\; r_{12}^{\prime}} + {D\; 2r_{302}^{\prime}}} & {{E\; 2\; r_{12}^{\prime}} + {F\; 2\; r_{32}^{\prime}}} & {{A\; 2\; r_{13}^{\prime}} + {B\; 2\; r_{33}^{\prime}}} & {{C\; 2\; r_{13}^{\prime}} + {D\; 2\; r_{33}^{\prime}}} & {{E\; 2\; r_{13}^{\prime}} + {F\; 2\; r_{33}^{\prime}}}\end{matrix} \rbrack}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}}\mspace{20mu}{{{A\; 1} = {{- \frac{f_{c}}{pw}}{ru}_{i}}},{{B\; 1} = {{ru}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{C\; 1} = {{- \frac{f_{c}}{pw}}{rv}_{i}}},\mspace{20mu}{{D\; 1} = {{rv}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{E\; 1} = {{- \frac{f_{c}}{pw}}f_{b}}},{{F\; 1} = {f_{b}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}}}\mspace{20mu}{{{A\; 2} = {{- \frac{f_{c}}{ph}}{ru}_{i}}},{{B\; 2} = {{ru}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{C\; 2} = {{- \frac{f_{c}}{ph}}{rv}_{i}}},\mspace{20mu}{{D\; 2} = {{rv}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{E\; 2} = {{- \frac{f_{c}}{ph}}f_{b}}},{{F\; 2} = {f_{b}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}}}} & ( {{Equation}\mspace{14mu} 21} )\end{matrix}$

Next, the first and second equations obtained for r₁₁˜r₃₃ are organizedas an integration equation (S1330). Thus, Equation 22 is acquired.

$\mspace{1499mu}{{{( {{Equation}\mspace{14mu} 22} )\mspace{20mu}\begin{bmatrix}{LW}_{1} \\{RW}_{1} \\\vdots \\{LW}_{n} \\{RW}_{n}\end{bmatrix}}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}$ $\mspace{20mu}{{LW}_{i} = \begin{bmatrix}{\frac{f_{c}}{pw}{lu}_{i}} & {\frac{f_{c}}{pw}{lv}_{i}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )u_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )v_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}{lu}_{i}} & {\frac{f_{c}}{ph}{lv}_{i}} & {\frac{f_{c}}{ph}f_{b}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )u_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )v_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )f_{b}}\end{bmatrix}}$ ${RW}_{i} = {\quad\lbrack \begin{matrix}{{A\; 1\; r_{11}^{\prime}} + {B\; 1\; r_{31}^{\prime}}} & {{C\; 1\; r_{11}^{\prime}} + {D\; 1\; r_{31}^{\prime}}} & {{E\; 1\; r_{11}^{\prime}} + {F\; 1\; r_{31}^{\prime}}} & {{A\; 1\; r_{12}^{\prime}} + {B\; 1\; r_{32}^{\prime}}} & {{C\; 1\; r_{12}^{\prime}} + {D\; 1\; r_{302}^{\prime}}} & {{E\; 1\; r_{12}^{\prime}} + {F\; 1\; r_{32}^{\prime}}} & {{A\; 1\; r_{13}^{\prime}} + {B\; 1\; r_{33}^{\prime}}} & {{C\; 1\; r_{13}^{\prime}} + {D\; 1\; r_{33}^{\prime}}} & {{E\; 1\; r_{13}^{\prime}} + {F\; 1\; r_{33}^{\prime}}} \\{{A\; 2\; r_{11}^{\prime}} + {B\; 2\; r_{31}^{\prime}}} & {{C\; 12r_{11}^{\prime}} + {D\; 2\; r_{31}^{\prime}}} & {{E\; 2\; r_{11}^{\prime}} + {F\; 2\; r_{31}^{\prime}}} & {{A\; 2\; r_{12}^{\prime}} + {B\; 2\; r_{32}^{\prime}}} & {{C\; 2\; r_{12}^{\prime}} + {D\; 2r_{302}^{\prime}}} & {{E\; 2\; r_{12}^{\prime}} + {F\; 2\; r_{32}^{\prime}}} & {{A\; 2\; r_{13}^{\prime}} + {B\; 2\; r_{33}^{\prime}}} & {{C\; 2\; r_{13}^{\prime}} + {D\; 2\; r_{33}^{\prime}}} & {{E\; 2\; r_{13}^{\prime}} + {F\; 2\; r_{33}^{\prime}}}\end{matrix} \rbrack}$

Next, the matrix [RL] is acquired by using such a method as singularvalue decomposition (SVD) (S1340).

More specifically, since the matrixes [LWi] and [RWi] contain twoequations, respectively, in Equation 22, a total of 4n equations areobtained for r11˜r33. Thus, they are acquired by using such a method assingular value decomposition (SVD).

Since Equation 22, which contains 4n equations, contains more equationsthan the method described in FIGS. 1 to 9, a more accurate result forthe matrix [RL] can be obtained. Thus, it is possible to make a moreaccurate posture measurement of the marker part 1110.

Now, the method of calculating the posture of the marker part 1110 bythe processing part 1130 will be described in more detail with referenceto the drawings.

FIG. 14 is a flowchart illustrating a method of calculating the postureof the marker part of an optical tracking system, according to anotherembodiment of the present invention.

Referring to FIG. 14, first, the processing part 1130 calibrates thefirst, second, third, and fourth conversion matrixes from three or moreimages (S1510).

The calibration may be substantially the same as operation S1200described in FIG. 11 and operations S1210 a and S1210 b to S1280described in detail in FIG. 12. The processing part 1130 may calibratethe first and second conversion matrixes by using only the finalequation for the calibration as in operations S1210 a, S1220 a, S1210 b,S1220 b, and S1280 among the operations above.

Next, the posture definition matrix is acquired from the first andsecond coordinate conversion formulas that contain the first, second,third, and the fourth conversion matrixes (S1520).

The acquisition of the posture definition matrix may be substantiallythe same as operation S1300 described in FIG. 11 and operations S1310 toS1340 described in detail in FIG. 13. The processing part 1130 mayacquire the posture definition matrix by using only the final equationfor the posture definition matrix as in operations S1330 and S1340 amongthe operations above.

Accordingly, the processing part 1130 may acquire: the first conversionmatrix for converting the first coordinate to the second coordinate; thesecond conversion matrix for converting the third coordinate to thefourth coordinate; the third conversion matrix for converting the fifthcoordinate to the sixth coordinate; and the fourth conversion matrix forconverting the seventh coordinate to the eighth coordinate through thecalibration in advance, and may acquire the posture definition matrixfor defining the posture of the marker part 1110 from the first andsecond coordinate conversion formulas.

Once the posture definition matrix is acquired, the posture of themarker part 1110 may be recognized. For example, the roll, pitch, andyaw of the marker part 1110 may be recognized from the posturedefinition matrix.

According to the optical tracking system as described above, in theoptical tracking system for tracking a marker part, the marker part canbe miniaturized while including a pattern of particular information toenable the tracking, and the posture of the marker part can bedetermined more accurately by modeling the optical systems of the markerpart and the image forming part with the coordinate conversion formulawhile applying a stereo structure thereto. Therefore, it is possible toaccurately track the marker part by a simpler and easier method.

Meanwhile, the optical tracking system 1000 as described above maydetermine the location of the marker part 1110, as well as the postureof the marker part 1110.

Hereinafter, a system modeling process for determining the location ofthe marker part 1110 and a process of determining the location of themarker part 1110 according to the same will be described in more detailwith reference to the drawings.

FIG. 15 is a flowchart schematically showing a problem-solving processthat is necessary for the processing part of the optical tracking systemof FIG. 10 to determine the location of the marker part.

Referring to FIG. 15, first, the system modeling of the optical trackingsystem 1000 that has the configuration described above is performed(S2100).

Unlike the coordinate conversion formula described above, the systemmodeling is performed so as to include the second coordinate (equal tothe sixth coordinate) corresponding to the coordinate of the center ofthe first lens 1114 (see FIG. 10) of the marker part 1110 (see FIG. 10)in order to acquire the location of the marker part 1110.

Subsequently, a location conversion matrix [T] is calibrated from theresult of the system modeling (S2200).

The location conversion matrix [T] refers to a matrix that converts thelocation of the first image forming part 1120 a (see FIG. 10) to thelocation of the second image forming part 1120 b (see FIG. 10) accordingto the separation of the first image forming part 1120 a and the secondimage forming part 1120 b.

Next, the location of the marker part 1110 is acquired (S2300).

More specifically, the three-dimensional coordinate of the center of thefirst lens 1114 of the marker part 1110 are acquired.

FIG. 16 is a conceptual diagram for explaining the process of systemmodeling in the problem-solving process of FIG. 15.

Referring to FIG. 16, the third coordinate conversion formula betweenthe second coordinate (P2) corresponding to the coordinate of the centerof the first lens 1114 of the marker part 1110 and the fourth coordinate(P4) corresponding to the first pixel coordinate on the first image ofthe pattern 1112 of the first image forming part 1120 a is configured.The third coordinate conversion formula may be expressed as a matrixequation as shown in Equation 23.

$\begin{matrix}{{s\begin{bmatrix}u_{1}^{\prime} \\v_{1}^{\prime} \\1\end{bmatrix}} = {{A_{L}\lbrack I \middle| 0 \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}} & ( {{Equation}\mspace{14mu} 23} )\end{matrix}$

Here, (u′,v′) denotes the fourth coordinate (P4). (X,Y,Z) denotes thesecond coordinate (P2). [AL] denotes the second conversion matrix. [I]denotes an identity matrix in a 3×3 form. In addition, [0] denotes azero-matrix in a 3×1 form. As shown in FIG. 16, since the second lens1122 a of the first image forming part 1120 a is the origin, based onthis, a rotation conversion portion of the fourth coordinate (P4)appears as an identity matrix, and a location conversion portion thereofappears as a zero-matrix. Thus, it may be expressed as [I|O].

Next, the fourth coordinate conversion formula between the sixthcoordinate (P6) {equal to the second coordinate (P2)} corresponding tothe coordinate of the center of the first lens 1114 of the marker part1110 and the eighth coordinate (P8) corresponding to the second pixelcoordinate on the second image of the pattern 1112 of the second imageforming part 1120 b is configured. The fourth coordinate conversionformula is configured so as to include the location conversion matrix[T] between the first image forming part 1120 a and the second imageforming part 1120 b. According to this, the fourth coordinate conversionformula may be expressed as a matrix equation as shown in Equation 24.

$\begin{matrix}{{s\begin{bmatrix}u_{2}^{\prime} \\v_{2}^{\prime} \\1\end{bmatrix}} = {{A_{R}\lbrack {R_{LR}❘T} \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}} & ( {{Equation}\mspace{14mu} 24} )\end{matrix}$

Here, (u′2,v′2) denotes the eighth coordinate (P8). (X,Y,Z) denotes thesixth coordinate (P6) {equal to the second coordinate (P2)}. [AR]denotes the fourth conversion matrix. [RLR] denotes a matrix that has a3×3 form and defines the posture of the first image forming part 1120 awith respect to the second image forming part 1120 b as described inFIG. 11 or the like. [T] denotes a location conversion matrix that has a3×1 form and converts the location of the first image forming part 1120a to the location of the second image forming part 1120 b. As shown inFIG. 16, since the first lens 1122 a of the first image forming part1120 a is the origin, based on this, a rotation conversion portion ofthe eighth coordinate (P8) of the second image forming part 1120 bappears as the matrix [RLR], and a location conversion portion thereofappears as [T]. Thus, it may be expressed as [RLR|T].

The factors shown in the system modeling are obtained in FIGS. 11 to 14above, except for the location conversion matrix [T]. Therefore, whenthe location conversion matrix [T] is obtained, the second coordinate(P2) {equal to the sixth coordinate (P6)} (i.e., the location of themarker part 1110) may be acquired.

Next, the operation S2200 of calibrating the location conversion matrix[T] among the third and fourth coordinate conversion formulas that areacquired as a result of the system modeling will be described in moredetail with reference to the drawings.

FIG. 17 is a flowchart illustrating a process of calibrating thelocation conversion matrix in the problem-solving process of FIG. 15.

Referring to FIG. 17, first, the first central coordinate the secondcentral coordinate are acquired, which are photographed in the first andsecond image forming units 1124 a and 1124 b, respectively (S2210).

Here, each of the first and second central coordinates refers to thecenter of a field of view that appears when photographing the pattern1112 in the first and second image forming units 1124 a and 1124 b,respectively, and a plurality of central coordinates may be acquired.

Next, a fundamental matrix [F] is calculated by using the first andsecond acquired central coordinates (S2220).

The fundamental matrix may be expressed as Equation 25.

$\begin{matrix}{{\lbrack {u_{2j}^{\prime}\mspace{14mu} v_{2j}^{\prime}\mspace{14mu} 1} \rbrack{F\begin{bmatrix}u_{1j}^{\prime} \\v_{1j}^{\prime} \\1\end{bmatrix}}} = {{{\lbrack {u_{2j}^{\prime}\mspace{14mu} v_{2j}^{\prime}\mspace{14mu} 1} \rbrack\begin{bmatrix}F_{11} & F_{12} & F_{13} \\F_{21} & F_{22} & F_{23} \\F_{31} & F_{32} & F_{33}\end{bmatrix}}\begin{bmatrix}u_{1j}^{\prime} \\v_{1j}^{\prime} \\1\end{bmatrix}} = 0}} & ( {{Equation}\mspace{14mu} 25} )\end{matrix}$

In Equation 25, the subscript j represents the order of data that isobtained when acquiring a plurality of central coordinates.

Equation 26 may be obtained by organizing Equation 25 with respect tothe components of the fundamental matrix [F]. With regard to Equation26, fundamental matrix [F], for example, may be acquired by applyingsuch a method as singular value decomposition (SVD) to eight or morepoints.

$\begin{matrix}{\begin{bmatrix}{u_{1j}^{\prime}u_{2j}^{\prime}} & {v_{1j}^{\prime}u_{2j}^{\prime}} & u_{2j}^{\prime} & {u_{1j}^{\prime}v_{2j}^{\prime}} & {v_{1j}^{\prime}v_{2j}^{\prime}} & v_{2j}^{\prime} & u_{1j}^{\prime} & v_{1j}^{\prime} & 1 \\\; & \; & \; & \vdots & \; & \; & \; & \; & \;\end{bmatrix}{\quad{\begin{bmatrix}F_{11} \\F_{12} \\F_{13} \\F_{21} \\F_{22} \\F_{23} \\F_{31} \\F_{32} \\F_{33}\end{bmatrix} = 0}}} & ( {{Equation}\mspace{14mu} 26} )\end{matrix}$

Subsequently, an essential matrix [E] is calculated by using theobtained fundamental matrix [F] (S2230).

The essential matrix [E] is expressed as shown in Equation 27.E=A _(R) ^(T) FA _(L)  (Equation 27)

Since the essential matrix [E] that is calculated from Equation 27 isexpressed as the product of a location conversion and a rotationconversion, when it is organized as shown in Equation 28, the matrix[t]X corresponding to the location conversion component may be obtained.

$\begin{matrix}{E = {\lbrack t\rbrack_{\times}R_{LR}}} & ( {{Equation}\mspace{14mu} 28} ) \\{\lbrack t\rbrack_{\times} = \begin{bmatrix}0 & {- t_{z}} & t_{y} \\t_{z} & 0 & {- t_{x}} \\{- t_{y}} & t_{x} & 0\end{bmatrix}} & \; \\{\lbrack t\rbrack_{\times} = {E \cdot R_{LR}^{- 1}}} & \;\end{matrix}$

Although the matrix [t]X may be the same as the location conversionmatrix, the location conversion matrix may be calculated by multiplyinga scale factor, if necessary.

Accordingly, next, the scale factor is acquired (S2240).

The scale factor may be acquired by measuring a marker from at least twolocations. For example, it may be calculated by moving and disposing themarker to at least two marker stages of which the separation distance isgiven and by measuring the same.

Then, the location conversion matrix [T] is acquired by using theacquired essential matrix [E] and the scale factor (S2250).

More specifically, the location conversion matrix [T] may be calculatedby multiplying the matrix (tx,ty,tz)T, which is a 3×1 matrix having thecomponents of tx, ty, and tz among the components of the acquired matrix[t]X, by the scale factor.

Next, the operation S2300 of acquiring the location of the marker part1110 by using the calibrated location conversion matrix [T] will bedescribed in more detail with reference to the drawings.

FIG. 18 is a flowchart illustrating an example of a process of acquiringthe location of a marker part in the problem-solving process of FIG. 15.

Referring to FIG. 18, first, the first equation is acquired from thefirst location conversion formula with respect to the first imageforming part 1120 a (S2310).

More specifically, Equation 23 that is the first location conversionformula is defined as shown in Equation 29.

$\begin{matrix}{{s\begin{bmatrix}u_{1}^{\prime} \\v_{1}^{\prime} \\1\end{bmatrix}} = {{{A_{L}\lbrack {I❘0} \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}} = {\begin{bmatrix}m_{1\_ 11} & m_{1\_ 12} & m_{1\_ 13} & m_{1\_ 14} \\m_{1\_ 21} & m_{1\_ 22} & m_{1\_ 23} & m_{1\_ 24} \\m_{1\_ 31} & m_{1\_ 32} & m_{1\_ 33} & m_{1\_ 34}\end{bmatrix}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}}} & ( {{Equation}\mspace{14mu} 29} )\end{matrix}$

The vector product of Equation 29 by itself on both sides results inzero to then be organized so that Equation 30 corresponding to the firstequation may be acquired.

$\begin{matrix}{\mspace{610mu}{{( {{Equation}\mspace{14mu} 30} )\lbrack \begin{matrix}{m_{{i\_}11} - {u^{\prime}m_{{i\_}31}}} & {m_{{i\_}12} - {u^{\prime}m_{{i\_}32}}} & {m_{{i\_}13} - {u^{\prime}m_{{i\_}33}}} & {m_{{i\_}14} - {u^{\prime}m_{{i\_}34}}} \\{m_{{i\_}21} - {v^{\prime}m_{{i\_}31}}} & {m_{{i\_}22} - {v^{\prime}m_{{i\_}32}}} & {m_{{i\_}23} - {v^{\prime}m_{{i\_}33}}} & {m_{{i\_}24} - {v^{\prime}m_{{i\_}34}}}\end{matrix} \rbrack}{\quad{\lbrack \begin{matrix}X \\Y \\Z \\1\end{matrix} \rbrack = 0}}}} & \;\end{matrix}$

Next, the second equation is acquired from the second locationconversion formula with respect to the second image forming unit 1120 b(S2320).

More specifically, the Equation 24 that is the second locationconversion formula is defined as shown in Equation 31.

$\begin{matrix}{{s\begin{bmatrix}u_{2}^{\prime} \\v_{2}^{\prime} \\1\end{bmatrix}} = {{{A_{R}\lbrack {R_{LR}❘T} \rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}} = {\begin{bmatrix}m_{2\_ 11} & m_{2\_ 12} & m_{2\_ 13} & m_{2\_ 14} \\m_{2\_ 21} & m_{2\_ 22} & m_{2\_ 23} & m_{2\_ 24} \\m_{2\_ 31} & m_{2\_ 32} & m_{2\_ 33} & m_{2\_ 34}\end{bmatrix}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}}} & ( {{Equation}\mspace{14mu} 31} )\end{matrix}$

The same operation as the operation S2310 of acquiring the firstequation is applied to Equation 31 in order to thereby acquire thesecond equation.

Equation 30 corresponding to the first equation includes two equations,and likewise, the second equation also includes two equations.Therefore, a total of four equations may be acquired to correspond totwo image forming units.

Next, the location of the marker part 1110 is acquired from the firstand second equations and the location conversion matrix [T] (S2330).

More specifically, the calibrated location conversion matrix [T] may beapplied to the first and the second equations in order to therebyacquire the three-dimensional coordinate of the center of the first lens1114 of the marker part 1110.

FIG. 19 is a flowchart illustrating another example of a process ofacquiring the location of a marker part in the problem-solving processof FIG. 15.

Referring to FIG. 19, first, the third equation is acquired from thefirst pattern coordinate conversion formula with respect to the firstimage forming unit 1120 a (S2350).

More specifically, the first pattern coordinate conversion formula isconfigured, which is transformed from the first location conversionformula, as shown in Equation 32.

$\begin{matrix}\begin{matrix}{{s\begin{bmatrix}u^{\prime} \\v^{\prime} \\1\end{bmatrix}} = {{{{A\lbrack {R❘t} \rbrack}\lbrack {R_{m}❘t_{m}} \rbrack}\begin{bmatrix}1 & 0 & 0 & {- u_{c}} \\0 & 1 & 0 & {- v_{c}} \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}}\begin{bmatrix}u \\v \\f_{b} \\1\end{bmatrix}}} \\{= \begin{bmatrix}m_{{i\_}11} & m_{{i\_}12} & m_{{i\_}13} & m_{{i\_}14} \\m_{{i\_}21} & m_{{i\_}22} & m_{{i\_}23} & m_{{i\_}24} \\m_{{i\_}31} & m_{{i\_}32} & m_{{i\_}33} & m_{{i\_}34}\end{bmatrix}} \\{\begin{bmatrix}r_{11} & r_{12} & r_{13} & X \\r_{21} & r_{22} & r_{23} & Y \\r_{31} & r_{32} & r_{33} & Z \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}{u - u_{c}} \\{v - v_{c}} \\f_{b} \\1\end{bmatrix}} \\{= \begin{bmatrix}m_{{i\_}11} & m_{{i\_}12} & m_{{i\_}13} & m_{{i\_}14} \\m_{{i\_}21} & m_{{i\_}22} & m_{{i\_}23} & m_{{i\_}24} \\m_{{i\_}31} & m_{{i\_}32} & m_{{i\_}33} & m_{{i\_}34}\end{bmatrix}} \\{\begin{bmatrix}{{r_{11}( {u - u_{c}} )} + {r_{12}( {v - v_{c}} )} + {r_{13}f_{b}} + X} \\{{r_{21}( {u - u_{c}} )} + {r_{22}( {v - v_{c}} )} + {r_{23}f_{b}} + Y} \\{{r_{31}( {u - u_{c}} )} + {r_{32}( {v - v_{c}} )} + {r_{33}f_{b}} + Z} \\1\end{bmatrix}}\end{matrix} & ( {{Equation}\mspace{14mu} 32} )\end{matrix}$

Equation 33 may be acquired by organizing Equation 32 as an equation.(m _(i) _(_) ₃₁ u′−m _(i) _(_) ₁₁)(u−u _(c))r ₁₁+(m _(i) _(_) ₃₁ u′−m_(i) _(_) ₁₁)(v−v _(c))r ₁₂+(m _(i) _(_) ₃₁ u′−m _(i) _(_) ₁₁)f _(b) r₁₃+(m _(i) _(_) ₃₁ u′−m _(i) _(_) ₁₁)X+(m _(i) _(_) ₃₂ u′−m _(i) _(_)₁₂)(u−u _(c))r ₂₁+(m _(i) _(_) ₃₂ u′−m _(i) _(_) ₁₂)(v−v _(c))r ₂₂+(m_(i) _(_) ₃₂ u′−m _(i) _(_) ₁₂)f _(b) r ₂₃+(m _(i) _(_) ₃₂ u′−m _(i)_(_) ₁₂)Y+(m _(i) _(_) ₃₃ u′−m _(i) _(_) ₁₃)(u−u _(c))r ₃₁+(m _(i) _(_)₃₃ u′−m _(i) _(_) ₁₃)(v−v _(c))r ₃₂+(m _(i) _(_) ₃₃ u′−m _(i) _(_) ₁₃)f_(b) r ₃₃+(m _(i) _(_) ₃₃ u′−m _(i) _(_) ₁₃)Z+(m _(i) _(_) ₃₄ u′−m _(i)_(_) ₁₄)=0(m _(i) _(_) ₃₁ v′−m _(i) _(_) ₂₁)(u−u _(c))r ₁₁+(m _(i) _(_) ₃₁ v′−m_(i) _(_) ₂₁)(v−v _(c))r ₁₂+(m _(i) _(_) ₃₁ v′−m _(i) _(_) ₂₁)f _(b) r₁₃+(m _(i) _(_) ₃₁ v′−m _(i) _(_) ₂₁)X+(m _(i) _(_) ₃₂ v′−m _(i) _(_)₂₂)(u−u _(c))r ₂₁+(m _(i) _(_) ₃₂ v′−m _(i) _(_) ₂₂)(v−v _(c))r ₂₂+(m_(i) _(_) ₃₂ v′−m _(i) _(_) ₂₂)f _(b) r ₂₃+(m _(i) _(_) ₃₂ v′−m _(i)_(_) ₂₂)Y+(m _(i) _(_) ₃₃ v′−m _(i) _(_) ₂₃)(u−u _(c))r ₃₁+(m _(i) _(_)₃₃ v′−m _(i) _(_) ₂₃)(v−v _(c))r ₃₂+(m _(i) _(_) ₃₃ v′−m _(i) _(_) ₂₃)f_(b) r ₃₃+(m _(i) _(_) ₃₃ v′−m _(i) _(_) ₂₃)Z+(m _(i) _(_) ₃₄ v′−m _(i)_(_) ₂₄)=0  (Equation 33)

Subsequently, the fourth equation is acquired from the second patterncoordinate conversion formula with respect to the second image formingunit 1120 b (S2360).

The present operation is substantially the same as the operation S2350of acquiring the third equation, except that the target is not the firstimage forming unit 1120 a but the second imaging forming unit 1120 b.

Equation 33 corresponding to the third equation includes two equations,and likewise, the fourth equation also includes two equations.Therefore, a total of four equations may be acquired to correspond totwo image forming units.

Next, the location of the marker part 1110 is acquired from the thirdand fourth equations and the location conversion matrix [T] (S2370).

More specifically, the calibrated location conversion matrix [T] may beapplied to the third and fourth equations in order to thereby acquirethe three-dimensional coordinate of the center of the first lens 1114 ofthe marker part 1110.

Hereinafter, a method for calculating the location of the marker part1110 by the processing part 1130 will be described in more detail withreference to the drawings.

FIG. 20 is a flowchart illustrating a method of calculating the locationof the marker part of the optical tracking system, according to anembodiment of the present invention.

Referring to FIG. 20, first, a plurality of coordinate values areacquired by photographing of the first and second image forming units1124 a and 1124 b (S2510).

The acquisition of the coordinate values is substantially the same asoperation S2210 of acquiring the first central coordinate and the secondcentral coordinate described in FIG. 17, and, for example, eightcoordinate values are acquired.

Next, a scale factor is acquired (S2520).

The acquisition of the scale factor is substantially the same asoperation S2240 described in FIG. 17.

Then, the location conversion matrix is calibrated by using the acquiredcoordinate values and scale factor (S2530).

The calibration may be substantially the same as operation S2200described in FIG. 15 and operation S2250 described in detail in FIG. 17.The processing part 1130 may calibrate the location conversion matrix byusing only the final equation for the calibration.

Next, the location of the marker part 1110 is acquired by using thecalibrated location conversion matrix (S2540).

The acquisition of the location of the marker part 1110 may besubstantially the same as operation S2300 described in FIG. 15,operations S2310 to S2330 described in detail in FIG. 18, or operationsS2350 to S2370 described in detail in FIG. 19. The processing part 1130may use only the final equation for the location acquisition as inoperation S2330 or S2370 among the operations above.

According to the optical tracking system as described above, since thelocation of the marker part can be determined more accurately bymodeling the marker part by applying a stereo structure, which isminiaturized while including a pattern of particular information, it ispossible to accurately track the marker part by a simpler and easiermethod.

Although the preferred embodiments of the invention have been describedin the detailed description of the invention, those skilled in the artor those who have ordinary knowledge in the art may modify and changethe present invention in various manners without departing from thespirit and scope of the present invention in the claims below.Therefore, the description above and the drawings below should beconstrued to show only examples of the present invention withoutlimiting the technical concept of the present invention.

REFERENCE NUMERALS

-   -   100, 1000: Optical tracking system 110, 1110: Marker part    -   112, 1112: Pattern 114, 1114: First lens    -   120, 1120: Image forming part 122, 1122: Second lens    -   124, 1124: Imaging forming unit 130, 1130: Processing part

What is claimed is:
 1. An optical tracking system comprising: a markerconfigured to include a pattern that has particular information and afirst lens that is spaced apart from the pattern and has a first focallength; a first image lens and sensor combination configured to includea second lens that has a second focal length and a first imaging sensorthat is spaced apart from the second lens and on which a first image ofthe pattern is formed by the first lens and the second lens; a secondimage lens and sensor combination configured to include a third lensthat has a third focal length and a second image imaging sensor that isspaced apart from the third lens and on which a second image of thepattern is formed by the first lens and the third lens; and a processorconfigured to determine a posture of the marker from a first coordinateconversion formula between a coordinate on a pattern surface of thepattern and a first pixel coordinate on the first image of the patternand from a second coordinate conversion formula between the coordinateon the pattern surface of the pattern and a second pixel coordinate onthe second image of the pattern, the second coordinate conversionformula including a rotation conversion between the first pixelcoordinate and the second pixel coordinate, and configured to track themarker.
 2. The optical tracking system according to claim 1, wherein theprocessor acquires: the first conversion matrix that converts a firstcoordinate corresponding to a coordinate on the pattern surface of thepattern to a second coordinate corresponding to a three-dimensionalcoordinate for the first lens of the marker; the second conversionmatrix that converts a third coordinate corresponding to athree-dimensional coordinate of the second coordinate for the secondlens to a fourth coordinate corresponding to the first pixel coordinateon the first image of the pattern of the first image lens and sensorcombination; the third conversion matrix that is the same as the firstconversion matrix and converts a fifth coordinate corresponding to acoordinate on the pattern surface of the pattern to a sixth coordinatecorresponding to a three-dimensional coordinate for the first lens ofthe marker; and the fourth conversion matrix that converts a seventhcoordinate corresponding to a three-dimensional coordinate of the sixthcoordinate for the third lens to an eighth coordinate corresponding tothe second pixel coordinate on the second image of the pattern of thesecond image lens and sensor combination, wherein the first coordinateconversion formula is defined to convert the first coordinate to thefourth coordinate while including the first conversion matrix and thesecond conversion matrix, and the second coordinate conversion formulais defined to convert the fifth coordinate to the eighth coordinatewhile including the third conversion matrix and the fourth conversionmatrix, and wherein the processor acquires, from the first coordinateconversion formula and the second coordinate conversion formula, a firstposture definition matrix that defines the posture of the marker withrespect to the first image lens and sensor combination.
 3. The opticaltracking system according to claim 2, wherein the first coordinateconversion formula is defined by ${{s\begin{bmatrix}{lu}^{\prime} \\{lv}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A_{l} \rbrack\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{lu} \\{lv} \\1\end{bmatrix}}},$ wherein {(lu,lv) denotes the first coordinate,(lu′,lv′) denotes the fourth coordinate, [C] denotes the firstconversion matrix, [A_(l)] denotes the second conversion matrix, [R_(L)]denotes the first posture definition matrix, and s denotes aproportional constant}, and the second coordinate conversion formula isdefined by ${{s\begin{bmatrix}{ru}^{\prime} \\{rv}^{\prime} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{R} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}} = {{{{\lbrack A_{r} \rbrack\lbrack R_{LR} \rbrack}\lbrack R_{L} \rbrack}\lbrack C\rbrack}\begin{bmatrix}{ru} \\{rv} \\1\end{bmatrix}}}},$ wherein {(ru,rv) denotes the fifth coordinate,(ru′,rv′) denotes the eighth coordinate, [C] denotes the thirdconversion matrix, [A_(r)] denotes the fourth conversion matrix, [R_(R)]denotes a second posture definition matrix for defining the posture ofthe marker with respect to the second image lens and sensor combination,[R_(LR)] denotes a third posture definition matrix for defining aposture of the first image lens and sensor combination with respect tothe second image lens and sensor combination, and s denotes aproportional constant}.
 4. The optical tracking system according toclaim 3, wherein the first conversion matrix and the third conversionmatrix are defined by ${\lbrack C\rbrack = \begin{bmatrix}1 & 0 & {- u_{c}} \\0 & 1 & {- v_{c}} \\0 & 0 & f_{b}\end{bmatrix}},$ wherein {(u_(c),v_(c)) denotes the coordinate of acenter of the pattern, and f_(b) denotes the first focal length}, andthe second conversion matrix and the fourth conversion matrix aredefined by ${\lbrack A\rbrack = \begin{bmatrix}{- \frac{f_{c}}{pw}} & 0 & u_{c}^{\prime} \\0 & {- \frac{f_{c}}{ph}} & v_{c}^{\prime} \\0 & 0 & 1\end{bmatrix}},$ wherein {(u′_(c),v′_(c)) denotes the pixel coordinateon the image of the pattern corresponding to the center of the pattern,f_(c) denotes the second focal length in the case of the secondconversion matrix and denotes the third focal length in the case of thefourth conversion matrix, pw denotes a width of a pixel of the firstimage of the pattern in the case of the second conversion matrix anddenotes a width of a pixel of the second image of the pattern in thecase of the fourth conversion matrix, and ph denotes a height of a pixelof the first image of the pattern in the case of the second conversionmatrix and denotes a height of a pixel of the second image of thepattern in the case of the fourth conversion matrix}.
 5. The opticaltracking system according to claim 4, wherein the processor acquires thefirst conversion matrix and the third conversion matrix by acquiringcalibration values of uc, vc, and fb from three or more photographedimages, and acquires the second conversion matrix and the fourthconversion matrix by acquiring calibration values of fc, pw, and ph byusing the acquired data.
 6. The optical tracking system according toclaim 3, wherein the processor acquires a plurality of pieces of data onthe first coordinate and the fourth coordinate and a plurality of piecesof data on the fifth coordinate and the eighth coordinate, and acquiresthe first posture definition matrix by the following equation to whichthe plurality of pieces of the acquired data are applied,$\begin{matrix}{{\lbrack R_{L} \rbrack = {{{\begin{bmatrix}r_{11} & r_{12} & r_{13} \\r_{21} & r_{22} & r_{23} \\r_{31} & r_{32} & r_{33}\end{bmatrix}\begin{bmatrix}{LW}_{1} \\{RW}_{1} \\\vdots \\{LW}_{n} \\{RW}_{n}\end{bmatrix}}\begin{bmatrix}r_{11} \\r_{12} \\r_{13} \\r_{21} \\r_{22} \\r_{23} \\r_{31} \\r_{32} \\r_{33}\end{bmatrix}} = 0}}{{LW}_{i} = \begin{bmatrix}{\frac{f_{c}}{pw}{lu}_{i}} & {\frac{f_{c}}{pw}{lv}_{i}} & {\frac{f_{c}}{pw}f_{b}} & 0 & 0 & 0 & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )u_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )v_{i}} & {( {{lu}_{i}^{\prime} - {lu}_{c}^{\prime}} )f_{b}} \\0 & 0 & 0 & {\frac{f_{c}}{ph}{lu}_{i}} & {\frac{f_{c}}{ph}{lv}_{i}} & {\frac{f_{c}}{ph}f_{b}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )u_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )v_{i}} & {( {{lv}_{i}^{\prime} - {lv}_{c}^{\prime}} )f_{b}}\end{bmatrix}}{{RW}_{i} = {{\quad\quad}\begin{bmatrix}{{A\; 1r_{11}^{\prime}} + {B\; 1r_{31}^{\prime}}} & {{C\; 1r_{11}^{\prime}} + \;{D\; 1r_{31}^{\prime}}} & {{E\; 1r_{11}^{\prime}} + \mspace{11mu}{F\; 1r_{31}^{\prime}}} \\{{A\; 1r_{12}^{\prime}} + {B\; 1r_{32}^{\prime}}} & {{C\; 1r_{12}^{\prime}} + \;{D\; 1r_{302}^{\prime}}} & {{E\; 1r_{12}^{\prime}} + \mspace{11mu}{F\; 1r_{32}^{\prime}}} \\{{A\; 1r_{13}^{\prime}} + {B\; 1r_{33}^{\prime}}} & {{C\; 1r_{13}^{\prime}} + \;{D\; 1r_{33}^{\prime}}} & {{E\; 1r_{13}^{\prime}} + \mspace{11mu}{F\; 1r_{33}^{\prime}}} \\{{A\; 2r_{11}^{\prime}} + {B\; 2r_{31}^{\prime}}} & {{C\; 2r_{11}^{\prime}} + \;{D\; 2r_{31}^{\prime}}} & {{E\; 2r_{11}^{\prime}} + \mspace{11mu}{F\; 2r_{31}^{\prime}}} \\{{A\; 2r_{12}^{\prime}} + {B\; 2r_{32}^{\prime}}} & {{C\; 2r_{11}^{\prime}} + \;{D\; 2r_{302}^{\prime}}} & {{E\; 2r_{12}^{\prime}} + \mspace{11mu}{F\; 2r_{32}^{\prime}}} \\{{A\; 2r_{13}^{\prime}} + {B\; 2r_{33}^{\prime}}} & {{C\; 2r_{13}^{\prime}} + \;{D\; 2r_{33}^{\prime}}} & {{E\; 2r_{13}^{\prime}} + \mspace{11mu}{F\; 2r_{33}^{\prime}}}\end{bmatrix}}}{{{A\; 1} = {{- \frac{f_{c}}{pw}}{ru}_{i}}},{{B\; 1} = {{ru}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{C\; 1} = {{- \frac{f_{c}}{pw}}{rv}_{i}}},{{D\; 1} = {{rv}_{i}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}},{{E\; 1} = {{- \frac{f_{c}}{pw}}f_{b}}},{{F\; 1} = {f_{b}( {{ru}_{c}^{\prime} - {ru}_{i}^{\prime}} )}}}{{{A\; 2} = {{- \frac{f_{c}}{ph}}{ru}_{i}}},{{B\; 2} = {{ru}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{C\; 2} = {{- \frac{f_{c}}{ph}}{rv}_{i}}},{{D\; 2} = {{rv}_{i}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}},{{E\; 2} = {{- \frac{f_{c}}{ph}}f_{b}}},{{F\; 2} = {{{f_{b}( {{rv}_{c}^{\prime} - {rv}_{i}^{\prime}} )}\lbrack R_{LR} \rbrack} = \begin{bmatrix}r_{11}^{\prime} & r_{12}^{\prime} & r_{13}^{\prime} \\r_{21}^{\prime} & r_{22}^{\prime} & r_{23}^{\prime} \\r_{31}^{\prime} & r_{32}^{\prime} & r_{33}^{\prime}\end{bmatrix}}},}} & \mspace{11mu}\end{matrix}$ wherein {(lu1,lv1), . . . , (lun,lvn) denote data of thefirst coordinate, (lu′1,lv′1), . . . , (lu′n,lv′n) denote data on thefourth coordinate, (lu′c,lv′c) denotes the pixel coordinate on the firstimage of the pattern corresponding to the center of the pattern,(ru1,rv1), . . . , (run,rvn) denote data on the fifth coordinate,(ru′1,rv′1), . . . , (ru′n,rv′n) denote data on the eighth coordinate,and (ru′c,rv′c) denotes the pixel coordinate on the second image of thepattern corresponding to the center of the pattern}.
 7. The opticaltracking system according to claim 3, wherein the processor determinesthe location of the marker from the third coordinate conversion formulafor the second coordinate and the fourth coordinate and from the fourthcoordinate conversion formula for the sixth coordinate and the eighthcoordinate, and tracks the marker by using the determined location ofthe marker.
 8. The optical tracking system according to claim 7, whereinthe third coordinate conversion formula is defined by${{s\begin{bmatrix}u_{1}^{\prime} \\v_{1}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A_{L} \rbrack\lbrack I\rbrack}\lbrack 0\rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}},$ wherein {(u′1,v′1) denotes the fourth coordinate,(X,Y,Z) denotes the second coordinate, [AL] denotes the secondconversion matrix, [I] denotes an identity matrix in a 3×3 form, [0]denotes a zero-matrix in a 3×1 form, and s denotes a proportionalconstant}, and the fourth coordinate conversion formula is defined by${{s\begin{bmatrix}u_{2}^{\prime} \\v_{2}^{\prime} \\1\end{bmatrix}} = {{{\lbrack A_{R} \rbrack\lbrack R_{LR} \rbrack}\lbrack T\rbrack}\begin{bmatrix}X \\Y \\Z \\1\end{bmatrix}}},$ wherein {(u′2,v′2) denotes the eighth coordinate,(X,Y,Z) denotes the sixth coordinate, [AR] denotes the fourth conversionmatrix, [RLR] denotes the third posture definition matrix in a 3×3 form,[T] denotes a location conversion matrix in a 3×1 form, and s denotes aproportional constant}.
 9. The optical tracking system according toclaim 8, wherein the processor: acquires a first central coordinate anda second central coordinate, each of which is the center of a field ofview of the pattern that is photographed in the first and second imagingsensors, respectively; calibrates the location conversion matrix betweenthe first image lens and sensor combination and the second image lensand sensor combination by using the acquired central coordinates; andacquires the location of the marker by using the calibrated locationconversion matrix.
 10. The optical tracking system according to claim 9,wherein the processor acquires a scale factor by measuring the marker attwo or more locations, and calibrates the location conversion matrixbetween the first image lens and sensor combination and the second imagelens and sensor combination by using the acquired scale factor togetherwith the acquired central coordinates.
 11. A method for calculating aposture and a location of a marker of an optical tracking system thatincludes a marker that includes a pattern that has particularinformation and a first lens that is spaced apart from the pattern andhas a first focal length, a first image lens and sensor combination thatincludes a second lens that has a second focal length and a firstimaging sensor that is spaced apart from the second lens and on which afirst image of the pattern is formed by the first lens and the secondlens, and a second image lens and sensor combination that includes athird lens that has a third focal length and a second imaging sensorthat is spaced apart from the third lens and on which a second image ofthe pattern is formed by the first lens and the third lens, and thatcalculates the posture of the marker for tracking the marker, the methodcomprising: acquiring a first conversion matrix that converts a firstcoordinate corresponding to a coordinate on a pattern surface of thepattern to a second coordinate corresponding to a three-dimensionalcoordinate for the first lens of the marker; a second conversion matrixthat converts a third coordinate corresponding to a three-dimensionalcoordinate of the second coordinate for the second lens to a fourthcoordinate corresponding to a first pixel coordinate on the first imageof the pattern of the first image lens and sensor combination, a thirdconversion matrix that is the same as the first conversion matrix andconverts a fifth coordinate corresponding to a coordinate on the patternsurface of the pattern to a sixth coordinate corresponding to athree-dimensional coordinate for the first lens of the marker, and afourth conversion matrix that converts a seventh coordinatecorresponding to a three-dimensional coordinate of the sixth coordinatefor the third lens to an eighth coordinate corresponding to a secondpixel coordinate on the second image of the pattern of the second imagelens and sensor combination; and acquiring a posture definition matrixfor defining the posture of the marker from a first coordinateconversion formula that converts the first coordinate to the fourthcoordinate while including the first conversion matrix and the secondconversion matrix and from a second coordinate conversion formula thatconverts the fifth coordinate to the eighth coordinate while includingthe third conversion matrix and the fourth conversion matrix.
 12. Themethod according to claim 11, further comprising: acquiring a firstcentral coordinate and a second central coordinate, each of which is acenter of a field of view of the pattern that is photographed in thefirst and second imaging sensors, respectively; calibrating the locationconversion matrix between the first image lens and sensor combinationand the second image lens and sensor combination by using the acquiredcentral coordinates; and acquiring the location of the marker by usingthe calibrated location conversion matrix.
 13. The method according toclaim 12, further comprising acquiring a scale factor by measuring themarker at two or more locations before calibrating the locationconversion matrix, wherein the location conversion matrix between thefirst image lens and sensor combination and the second image lens andsensor combination is calibrated by using the acquired scale factortogether with the acquired central coordinates in the calibrating of thelocation conversion matrix.